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STATEMENT OF FACTS 

ACCOMPANYING A PETITION UNDER 37 CFR 1.47(B) TO ACCEPT AN UNSIGNED 
SUPPLEMENTAL REISSUE DECLARATION BASED ON A REFUSAL OF ALL OF 

THE INVENTORS 

Sir: 

1. I, Michael L. Taylor, Reg. No. 50,521 make the following "Statement of 
Facts" regarding communications and refusals of the inventors of U.S. Pat. App. No. 
09/784,829, which is a reissue application of U.S. Patent 5,568,675, to sign and return 
for filing with the U.S. Patent and Trademark Office a Supplemental Reissue 
Declaration. 



2. On August 29, 2011, I sent an email to inventor Michel Scriban requesting 
a time to speak with him via telephone regarding the subject reissue application 
09/784,829 in which he is a co-inventor with Joel Henrion, Jean Francois Uhl, and Jean- 
Baptiste Thiebeaut. After the initial email on August 29, 2011 (See, Attachment A), I 
had a brief telephone conversation, in the English language, with Michel Scriban on 
August 30, 2011 to discuss the requirement for a Supplemental Reissue Declaration 
and the best way to send the necessary papers to Michel Scriban and all of his co- 
inventors. 

3. On August 31, 2011, I sent a first email communication with papers for 
review to the last known email addresses for Michel Scriban, Joel Henrion, and Jean 
Francois Uhl. In this first email I also requested that Michel Scriban contact Jean- 
Baptiste Thiebeaut In this first email I included a Supplemental Reissue Declaration in 
both French and English languages, the U.S. Patent 5,568,675 for which the reissue is 
being requested, and the currently pending claims. See, Attachment B. This first email 
sent on August 31, 2011 was sent in English and French for convenience of the 
inventors who we believed were domiciled in France. An English original of the French 
Language portion of the email is included under Attachment C. 

4. Included under Attachments D-F (where Attachment F includes an English 
translation of the original French communication under Attachment E) are various 
communications between Michel Scriban and myself during the month of September 
2011. 

5. Following the communications between Michel Scriban and myself in 
September 2011, Michel Scriban made a first demand in an email to me on October 9, 
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201 1 for payment of 70,800 EUROS (about US$95,500) to be paid to all of the inventors 
for their review of the papers sent in my first email of August 31, 2011 and to execute 
the Supplemental Reissue Declaration. See, Attachment G. An English language 
translation of the letter under Attachment Tab G is provided under Attachment H. 

6. In response to Michel Scriban's October 9, 2011 email (Attachment G), 
Richard W. Warner, Reg. No. 38,043 and I prepared a detailed letter, sent on November 
1, 2011, setting out the current Assignee's position that no further payments were 
required and that such a demand of payment to review and execute the Supplemental 
Reissue Declaration amounted to a refusal to sign. The November 1, 2011 letter was 
sent both via email to each of the inventors and via DHL courier service. See, 
Attachment I. The November 1 letter included attachments of the original Assignment 
from each of the inventors to Daidix S.A. that indicated that further remuneration was 
not necessary to assist in obtaining reissues of the original application. Further, we 
included the Assignment recordal chain from the www.uspto.gov website indicating 
assignments of the U.S. Patent 5,568,675 to Medtronic, Inc. Additionally, we included a 
paper copy of the Supplemental Reissue Declaration, both in English and French, the 
originally issued U.S. Patent 5,868,675 of which this is a reissue application, and the 
claims as currently pending. 

7. We received confirmation from DHL that the package was delivered to and 
accepted at each of the addresses that we had for all four of the named inventors of the 
U.S. Patent 5,868,675. I sent an email to each of the inventors noting that we had 
confirmation from DHL, including a copy of the confirmations, that each of the inventors 
had received the package. See, Attachment J. Joel Henrion indicated that the package 
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was not sent to his proper address and that he did not receive it. However, Joel 
Henrion communicated this in an email (See, Attachment K) that originated from the 
email address to which I also sent the same package of information (including the 
Supplemental Reissue Declaration, the originally issued U.S. Patent 5,568,675, and the 
currently pending claims). Accordingly, I believe that Joel Henrion did receive an email 
communication that included all of the same papers that were sent via the DHL courier 
to the address I had for Joel Henrion. Prior to resending a paper copy to Joel Henrion, I 
was notified that an attorney Maxime Grange in France now represents all of the 
inventors and that the attorney has also received the communication sent on November 
1, 2011 that included the Supplemental Reissue Declaration, the issued U.S. Patent 
5,568,675, and the claims as pending. See, Attachment L. 

8. We requested in our letter of November 1, that each of the inventors 
execute and return to Richard W. Warner (or myself) the Supplemental Reissue 
Declaration by November 14, 2011. We provided this extended time for reply even 
though each of the inventors had previously been forwarded the package, at least as 
early as August 31 st , either directly by me or through Michel Scriban. Nevertheless, on 
November 15, 2011, I received an email from attorney Maxime Grange indicating that 
the inventors did not intend to sign the Supplemental Reissue Declaration and that, in 
addition to the previously requested monies, an additional 4200 EUROS (about 
US$5600) was required for his services in setting up of an escrow account to receive 
the previously demanded 70,800 EUROS (about US$95,500). See, Attachment L. An 
English translation of the letter from Attorney Grange is provided under Attachment M. 
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9. I responded to attorney Maxime Grange's letter on November 23, 201 1 by 
email including a request to confirm that attorney Maxime Grange was an attorney for 
all of the inventors Michel Scriban, Joel Henrion, Jean Francois Uhl, and Jean-Baptiste 
Thiebeaut I additionally forwarded to attorney Grange my November 1, 2011 letter 
originally sent to all of the inventors that included the Supplemental Reissue 
Declaration, a copy of the US Patent 5,868,675, a copy of the claims as currently 
pending, the assignment chain from www.uspto.gov, and the assignment from all of the 
inventors to Daidix S.A. See, Attachment N. 

10. Finally, on November 29, 2011, I received an email from attorney Grange 
following up to my email of November 23, stating that attorney Grange had received my 
correspondences to which he was replying in the email. He further states that he 
represents the French experts and that they cannot study the documents that we 
provided and it is not possible to sign them. See, Attachment O, an English Translation 
of the email is under Attachment P. 



P.O. Box 828 

Bloomfieid Hills, Michigan 48303 
(248) 641-1600 

RWW/MLT/srh 

16429102.1 
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ATTACHMENT A 



Taylor, Michael 



From: Taylor, Michael <mltaylor@HDP.com> 

Sent: Monday; August 29, 2011 3:21 PM 

To: m.scriban@nelixa.fr 

Cc: Warner ; Rick 

Subject: Re-issue of U.S. Patent 5,568,675 (5074a-000013/REA) 



Dear Dr. Scriban, 



You may recall in late 2002 you communicated with Richard Warner regarding the above referenced U.S. Patent. 



We are nearing the end of prosecution, and I was wondering if I could call you tomorrow at 33-4-72-7 1-01-39 at about 
2:30PM Paris time regarding contact information for you and your co-inventors Joel Henrion, Jean Francois UHL, and 
Jean-Baptiste Thiebaut. 



Thank you, 



Michael 



Michael L. Taylor °f f,ce: ? 4 ! f 4 J 

i p , ,. Alt Direct: 248.641.1289 

i ait.ru ACiornty Fax; 248.641,0270 



Hi^l-^lfSl ESS Iffl Mi 5445 Corporate Dr, Suite 

if ^^iM^sa B ^ ir""! 1 i - "**"* jcr^NLy 200 
B ^MMM LJIL^t^fc^Y Troy, MI 48098 

| IP Causes Worldwide Bio ♦ vCard 

j Metropolitan j Detroit ♦ Portland ♦ St. Louis ♦ Washington, DC 



HARNESS, DICKEY & PBERCE, PLC LEGAL NOTICE: The information contained in this transmission is intended only for the individuai(s) or entity(ies) to 
whom it is addressed. It may contain information protected from use and/or disclosure by law, including information that is protected as confidential, attorney-client 
privileged, attorney work product and/or trade secrets. If the reader of this message is not the intended recipient, or an employee or agent responsible for 
delivering this message to the addressee, the reader is hereby notified that any use, distribution or copying of this communication is strictly prohibited, if you 
believe you have received this facsimile or message in error, please immediately notify us at. our expense by return mail or e-mail and permanently delete or 
destroy all copies of the message. 

To comply with U.S. Treasury regulations, we advise you that any discussion of Federal tax issues in this communication was not intended to be used, and 
cannot be used, by any person (i) to avoid penalties that may be imposed by the interna! Revenue Service, or (ii) to promote, market or recommend to another 
party any matter addressed herein. 

Any portions of this transmission containing controlled technical data are restricted by U.S. export laws and regulations, and may not be distributed or 
retransmitted to non-U. S. persons without appropriate licensing or a licensing exemption. Neither this information block, nor the signature block, nor the typed 
name of the sender, nor anything else in this message is intended to constitute an electronic signature unless a specific statement to the contrary is included. 
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ATTACHMENT B 



Taylor, Michael 



From: 
Sent: 
To: 
Cc: 



Subject: 
Attachments: 



Taylor; Michael <mltaylor@HDP.com> 
Wednesday, August 31, 2011 3:06 PM 
m.scriban@nelixa.fr 

Joel.henrion@wanadooir; Jf.uhl@free.fr; Jf.uhl@wanadoo.fr; Warner, Rick; Neal, Patrick; 
Hall, Stephanie 

Re-issue of U.S. Patent 5,568,675 (5074a-000013/REA) 

Supplemental Reissue declaration.PDF; US 5868675.PDF; Current Claims.DOC 



Dr. Scriban, 



As we discussed yesterday, I am forwarding this email that discusses the U.S. Patent Application that is a reissue of the 
above referenced U.S. Patent and papers that we need executed. I am also copying each inventor for which we 
previously had an email address, Joel Henrion and Joel Francois Uhl. You indicated that you would forward this email 
to Jean-Baptiste Thiebaut, and also to Joel Henrion and Joel Francois Uhl if the email addresses we have for them are no 
longer correct. We previously had translated the below message that discusses the current process and our reason for 
contacting you. Also, if you have an email address for Jean-Baptiste Thiebaut, we would very much appreciate receiving 
it from you. 

We very much appreciate your assistance in this matter. Please let us know any one of you will not be able to 
return the executed paper to us by October 3, 201 1 . 



Comme vous pouvez vous en souvenir, vous avez ete en contact avec Rick Warner ou Christopher Eusebi fin 2002 et 
debut 2003 concernant une demande de redelivrance du brevet U.S. 5 568 675 (ci-joint). A ce moment-la, vous aviez 
execute une declaration de redelivrance. 



Lors de votre travail avec le bureau des brevets U.S. pour obtenir un brevet subventionnable, nous avons amende les 
reclamations par celles jointes dans les « reclamations actuelles » ci-jointes, que nous vous soumettons pour 
revision. Selon les regies regissant les demandes de redelivrance, nous allons deposer une declaration de redelivrance 
additionnelle. Nous avons joint une « declaration de redelivrance additionnelle » pour votre execution. Nous pensons qu'il 
s'agit du dernier document necessaire pour cette demande. Une fois que le bureau des brevets U.S. re?oit ce document 
dGment execute, une redelivrance de brevet devrait etre accordee. Un brevet redelivre remplace essenttellement le brevet 
d'origine. 



Veuillez dument executer la « declaration de redelivrance additionnelle » et nous la renvoyer. Vous pouvez nous 
I'envoyer par courriel ou par fax. Si vous ne pouvez nous envoyer pas courriel ou par fax une copie dQment executee et 
devez envoyer une copie physique, nous pouvons vous donner un numero de compte DHL pour couvrir le cout de 
1'envoi. 



Dans le tableau ci-dessous, vous trouverez les coordonnees que nous avons pour chacun des inventeurs. Veuillez 
confirmer que ces informations sont exactes. Si vous pouviez egalement nous donner Padresse courriel de Jean-Baptiste 
Thiebaut, nous en serions tres reconnaissants. 



l 



Nous vous remercions d'avance pour votre aide et votre reponse rapide. 



Name 


DHL/Mailina Address 


Email Addressees ^ 


Phone 


Michel Scriban 


72 Chemin de Crapon 
69360 Ternay, France 


m.scriban@nelixa.fr 




Joel Henrion 


17, Route de Chalone 
51600 Suippes, France 


Joel.henrion@wanadooir 




Jean Francois UHL 


199 avenue du Maine 
Paris, France 75014 

(auxiliary address- 12 rue Regard; 92380 Garche, 
France) 


Jf.uhlfi8free.fr 
Jf.uhl@wanadoo.fr 




Jean-Baptiste Thiebaut 


42 boulevard Saint-Marcel 
Paris, France 75005 







Thank you and Best Regards, 



Michael 



Michael L. Taylor office: 248.64i.i6oo 

Patent Attorney Direct: 248.641.1289 

y Fax: 248.641.0270 



M>^!^IME»SS §ilil 5445 Corporate Dr, Suite 
□ ICKEY 200 



Troy, MI 48098 
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English 


French 


SUPPLEMENTAL DECLARATION 

FOR REISSUE 

PATENT APPLICATION 

TO CORRECT "ERRORS" STATEMENT 

(37 CFR 1.175) 


DECLARATON SUPPLEMENTAIRE POUR REDELIVRANCE 
D'UNE DEMNADE DE BREVET POUR CORRIGER DES 
« ERREURS » 
(37 CFR 1.175) 






Attorney Docket: 5074A-000013/REA 
First Named Inventor: Jean Francois Uhl 
Application Number: 09/784,829 
Filing Date: February 8, 2001 
Art Unit: 3737 

Examiner Name: Ruth S. Smith 


Numero de registre: 5074A-000013/REA 
Norn du premier inventeur: Jean Francois Uhl 
Numero de ['application: 09/784,829 
Date de depot: 8 fevrier 2001 
Unite d'art : 3737 

Norn de Pexaminateur; Ruth S. Smith 




I/We hereby declare that: 

Every error in the patent which was corrected in the present reissue 
application, and which is not covered by the prior oath(s) and/or 
declaration(s) submitted in this application, arose without any 
deceptive intention on the part of the applicant. 

WARNING: 

Petitioner/applicant is cautioned to avoid submitting personal 
information in documents files in a patent application that may 
contribute to identity theft. Persona! information such as social 
security numbers, bank account numbers, or credit card numbers 
(other than a check or credit card authorization form PTO-2038 
submitted for payment purposes) is never required by the USPTO to 
support a petition or an application, if this type of personal 
information is included in documents submitted to the USPTO, 
petitioners/applicants should consider redacting such personal 

inf nrmatinn f mm f Ho Hon i m onf c hnfnro eiihmittinrt t Horn to tho 
ii tiui if iciuui 1 it urn ii its uutj-uu ici n» ut: iui c auuii 11 limy u mm iu iiic 

USPTO. 

Petitioner/application is advised that the record of a patent 
application is available to the public after publication of the 
application (unless a non-publication request in compliance with 37 
CFR 1.213(a) is made in the application) or issuance of a patent. 
Furthermore, the record from an abandoned application may also be 
available to the public if the application is referenced in a published 
application or an issued patent (see 37 CFR 1.14). Checks and 
credit card authorization forms PTO-2038 submitted for payment 
purposes are not retained in the application file and therefore are 
not publicly available. 

I/We hereby declare that all statements made herein of my/our own 
knowledge are true and that all statements made on information and 
belief are believed to be true; and further that these statements were 
made with the knowledge that willful false statements and the like 
so made are punishable by fine or imprisonment, or both, under 18 
U.S.C. 1001 and that such willful false statements may jeopardize 
the validity of the application or any patent issued thereon. 


Je(nous) declare(ons) que : 

Toute erreur dans le brevet qui a ete corrigee dans cette demande de redelivrance 
et qui n'est pas couverte par le present serment ou declaration soumis a cette 
demande, s'est produlte sans intention de tromper de la part de 1'appliquant. 

AVERTISSEMENT : 

Le requerant / demandeur est mis en garde contre la soumission de 
renseignements personnels dans tes documents deposes dans une demande de 
brevet qui peuvent aider a ('usurpation d'identite. Les renseignements personnels 
tels que numeros de securite sociale, numeros de compte bancaire, ou numeros 
de carte de credit (autre qu'un cheque ou un formulaire d'autorisation de carte de 
credit PTO-2038 dans le but de faire un paiement) ne sont jamais requis par 
I'USPTO (Bureau des brevets des Etats-Unis) pour appuyer une petition ou une 
demande. SI ce type de renseignements personnels est inclus dans les documents 
deposes a I'USPTO, les demandeurs / requerants devraient envisager de les 
enlever des documents avant de les soumettre a I'USPTO. 

le requerant / demandeur est informe que le dossier de demande de brevet est a la 
disposition du public apres la publication de la demande (sauf si une demande de 
non-publication en conformite avec 37 CFR 1.213 (a) a ete faite dans cette 
demande) ou apres la deiivrance d'un brevet. En outre, le dossier d'une demande 
abandonnee peut egalement etre mis a la disposition du public si la demande est 
referencee dans une application publiee ou un brevet delivre (voir 37 CFR 1.14). 
Les cheques et formulaires d'autorisation de carte de credit PTO-2038 soumis pour 
fins de paiement ne sont pas conserves dans le dossier de demande et ne sont 
done pas accessibles au public. 

Je / Nous declarons que toutes les declarations faites selon ma/notre 
connalssance dans ce document sont veridiques et que toutes les declarations 
faites sur des informations et croyances sont considerees comme vraies. De plus, 
ces declarations ont ete faites en sachant que toute fausse declaration volontaire 
est passible d'une amende ou d'une peine d'emprisonnement, ou les deux, en 
vertu de la loi 18 USC 1001 et que de telles declarations risquent de compromettre 
la validite de la demande ou d'un brevet delivre a partlr de celle-ci. 


Name of First Inventor: Jean Francois Uhl 
Inventor's Sianature 


Norn du premier inventeur: Jean Francois Uhl 
Sianature de Pinventeur 


Date 


Date 








Name of Second Inventor: Joel Henrion 
Inventor's Signature 


Nom du deuxieme inventeur : Joel Henrion 
Sianature de Pinventeur 


Date 


Date 








Name of Third Inventor: Michel Scriban 
Inventor's Sianature 


Nom du troisieme inventeur : Michel Scriban 
Sianature de Pinventeur 


Date 


Date 








Name of Fourth Inventor: Jean-Baptiste Thiebaut 
Inventor's Sianature 


Nom du quatrieme inventeur : Jean-Baptiste Thiebaut 
Sianature de Pinventeur 


Date 


Date 
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United States Patent [19] 

Henrion et al. 
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US005868675A 
[ii] Patent Number: 
[45] Date of Patent: 



5,868,675 
*Feb. 9, 1999 



[54] INTERACTIVE SYSTEM FOR LOCAL 
INTERVENTION INSIDE A 
NONHUMOGENEOUS STRUCTURE 

[75] Inventors: Joel Henrion, Suippes; Michel 
Seriban, Ternay; Jean-Baptiste 
Thiebaut; Jean-Francois Uhl, both of 
Paris, all of France 

[73] Assignee: Elekta IGS SA., Gieres, France 

[ * ] Notice: The terminal 36 months of this patent has 
been disclaimed. 

[21] AppL No.: 847,059 

[22] PCT Filed: May 10, 1990 

[86] PCT No,: PCT/FR90/00714 

§371 Date: Jun. 22, 1992 

§ 102(e) Date: Jun. 22, 1992 
[87] PCT Pub. No.: WO91/04710 

PCT Pub. Date: Apr. 18, 1991 

[30] Foreign Application Priority Data 

Oct. 5, 1989 [FR] France 89 13028 

[51] Int. CI. 6 A61B 5/05 

[52] U.S. CI. 600/424; 606/130 
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378/20, 41, 58, 205; 606/130; 901/6, 16, 
41; 600/407, 411, 415, 417, 424 
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ABSTRACT 



An interactive system for a local intervention inside a region 
of a non-homogeneous structure, such as the skull of a 
patient, which is related to the frame of reference (R^ of an 
operation table, and which is connected to a reference 
structure comprising a plurality of base points. The system 
creates on a screen a representation of the non-homogeneous 
structure and of the reference structure connected thereto, 
provides the coordinates of the images of the base points in 
the first frame of reference (Rj), allows the marking of the 
coordinates of the base points in R 2 , and allows the carrying 
out of the local intervention with an active member such as 
a trephining tool, a needle, or a radioactive or chemical 
implant. The systems also optimizes the transfer of reference 
frames between R x and R 2 , from the coordinates of the base 
points in R 2 and the images in R a by reducing down to a 
minimum the deviations between the coordinates of images 
in R 2 and the base points in R 2 after transfer, The system also 
establishes real time bi-directional coupling between: (1) an 
origin and a direction of intervention simulated on the 
screen, (2) the position of the active member. 

16 Claims, 13 Drawing Sheets 
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INTERACTIVE SYSTEM FOR LOCAL 

INTERVENTION INSIDE A 
NONHUMOGENEOUS STRUCTURE 

The invention relates to an interactive system for local 5 
intervention inside a region of a nonhomogeneous structure. 

The performing of local interventions inside a nonho- 
mogeneous structure, such as intracranial surgical opera- 
tions or orthopedic surgery currently poses the problem of 
optimizing the intervention path or paths so as to secure, on 
the one hand, total intervention over the region or structure 
of interest, such as a tumor to be treated or explored and, on 
the other hand, minimal lesion to the regions neighboring or 
adjoining the region of interest, this entailing the localizing 
and then the selecting of the regions of the nonhomogeneous 
structure which are least sensitive to being traversed or the 15 
least susceptible to damage as regards the integrity of the 
structure. 

Numerous works aimed at providing a solution to the 
abovementioned problem have hitherto been the subject of 
publications. Among the latter may be cited the article 20 
entitled "Three Dimensional Digitizer (Neuron avigator): 
New Equipment for computed Tomography Guided Stereo- 
taxic Surgery", published by Eiju Watanabe, M.D., Takashi 
Watanabe, M.D., Shinya Manaka, M.D., Yoshiaki 
Mayanagi, MD., and Kinlorno Takakura, M.D. Department 25 
of Neurosurgery, Faculty of Medicine, University of Tokyo, 
Japan, in the journal Surgery Neurol. 1987: 27 pp. 543-547, 
by Elsevier Science Publishing Co., Inc. The Patent WO-A- 
88 09151 teaches a similar item of equipment. 

In the abovementioned publications are described in 30 
particular a system and an operational mode on the basis of 
which a three-dimensional position marking system, of the 
probe type, makes it possible to mark the three-dimensional 
position coordinates of a nonhomogeneous structure, such as 
the head of a patient having to undergo a neurosurgical 35 
intervention, and then to put into correspondence as a 
function of the relative position of the nonhomoc.eneous 
structure a series of corresponding images consisting of 
two-dimensional images sectioned along an arbitrary 
direction, and obtained previously with the aid of a medical 40 
imaging method of the "scanner" type. 

The system and the operational mode mentioned above 
offer a sure advantage for the intervening surgeon since the 
latter has available, during the intervention, apart from a 
direct view of the intervention, at least one two-dimensional 45 
sectional view enabling him to be aware, in the sectional 
plane, of the state of performance of the intervention. 

However, and by virtue of the very design of the system 
and of the operational mode mentioned above, the latter 
allow neither a precise representation of the state of perfor- 50 
rnance of the intervention, nor partially or totally automated 
conduct of the intervention in accordance with a program for 
advance of the instrument determined prior to the interven- 
tion. 

Such a system and such an operational mode cannot 55 
therefore claim to eradicate all man-made risk, since the 
intervention is still conducted by the surgeon alone. 

The objective of the present invention is to remedy the 
whole of the problem cited earlier, and in particular to 
propose a system permitting as exact as possible a 60 
correlation, at any instant, between an intervention modeling 
on the screen and the actual intervention, and furthermore 
the representation from one or more viewing angles, and if 
appropriate in one or more sectional planes, of the nonho- 
mogeneous structure, the sectional plane or planes possibly 65 
being for example perpendicular to the direction of the path 
of advance of the instrument or of the intervention tool. 



Another objective of the present invention is also the 
implementation of a system permitting simulation of an 
optimal trajectory of advance of the tool, so as to constitute 
an assisted or fully programed intervention. 

Finally, an objective of the present invention is to pro- 
pose a system making it possible, on the basis of the 
simulated trajectory and of the programed intervention, to 
steer the movement of the instrument or tool to the said 
trajectory so as to carry out the programed intervention. 

The invention proposes to this effect an interactive 
system for local intervention inside a region of a nonhomo- 
geneous structure to which is tied a reference structure 
containing a plurality of base points, characterized in that it 
comprises: 

means of dynamic display by three-dimensional imaging 
of a representation of the nonhomogeneous structure 
and of a reference structure tied to the nonhomoge- 
neous structure, including images of the base points, 
means of delivering the coordinates of the images of the 

base points in the first reference frame, 
means of securing the position of the non-homogeneous 
structure and the reference structure with respect to a 
second reference frame, 
marker means for delivering the coordinates of the base 

points in the second reference frame, 
means of intervention comprising an active member 
whose position is determined with respect to the second 
reference frame, 
means of optimizing the transfer of reference frames from 
the first reference frame to the second reference frame 
and vice versa, on the basis of the coordinates of the 
images of the base points in the first reference frame 
and of the coordinates of the base points in the second 
reference frame, in such a way as to reduce to a 
minimum the deviations between the coordinates of the 
images of the base points in the first reference frame 
and the coordinates of the base points, expressed in the 
said first reference frame with the aid of the said 
reference frame transfer tools, 
means for defining with respect to the first reference 
frame a simulated origin of intervention and a simu- 
lated direction of intervention, and 
reference frame transfer means using the said reference 
frame transfer tools to establish a bidirectional coupling 
between the simulated origin of intervention and the 
simulated direction of intervention and the position of 
the active member. 
A more detailed description of the system of the invention 
will be given below with reference to the drawings in which: 
FIG. 1 represents a general view of an interactive system 
for local intervention inside a region of a nonhomogeneous 
structure according to the present invention, 

FIG. 2 represents, in the case where the nonhomogeneous 
Structure consists of the head of a patient, and with a view 
to a neurosurgical intervention, a reference structure tied to 
the nonhomogeneous structure and enabling a correlation to 
be established between a "patient" reference frame and a 
reference frame of images of the patient which were made 
and stored previously, 

FIG. 3 represents an advantageous embodiment of the 
spacial distribution of the reference structure of FIG. 2, 

FIG. 4 preesents an advantageous embodiment of the 
intervention means set up on an operating table in the case 
of a neurosirgical intervention, 

FIGS. Set and Sb represent a general flow diagram of 
functional steps implemented by the system, 
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FIGS, 6 thru 8 represent How diagrams of programs 
permitting implementation of certain functional steps of 
FIG. 56, 

FIG. 9a represents a flow diagram of a program permit- 
ting implementation of a functional step of FIG. Sa, 5 

FIG. 9b represents a flow diagram of a program permit- 
ting implementation of another functional step of FIG, 5a, 

FIGS. 10a and 106 represent a general flow diagram of 
the successive steps of an interactive dialogue between the 
system of the present invention and the intervening surgeon 10 
and 

FIG. 10c represents a general flow diagram of the suc- 
cessive functional steps carried out by the system of the 
invention, having (sic) the intervention, prior to the 
intervention, during the intervention and after the interven- 15 
tion. 

The interactive system for local intervention according to 
the invention will firstly be described in connection with 
FIG. 1. 

A nonhomogeneous structure, denoted SNH, on which an 20 
intervention is to be performed, consists for example of the 
head of a patient in which a neurosurgical intervention is to 
be performed. It is however understood that the system of 
the invention can be used to carry out any type of interven- 
tion in any type of nonhomogeneous structure inside which 25 
structural and/or functional elements or units may be in 
evidence and whose integrity, during the intervention, is to 
be respected as far as possible. 

The system comprises means, denoted 1, of dynamic 
display by three-dimensional imaging, with respect to a first 30 
reference frame R l5 of a representation (denoted RSR) of a 
reference structure SR (described later) tied to the structure 
SNH, and a representation or modeling of the nonhomoge- 
neous structure, denoted RSNH. 

More precisely, the means 1 make it possible to display a 35 
plurality of successive three-dimensional images, from dif- 
ferent angles, of the representations RSNH and RSR. 

The system of the invention also comprises means, 
denoted 2, of tied positioning, with respect to a second 
reference frame R 2 , of the structures SNH and SR. 40 

In the present non-limiting example, the head of the 
patient, bearing the reference structure SR, is fixed on an 
operating table TO to which are fixed the means 2 of tied 
positioning. 

Of course, the patient whose head has been placed in the 45 
means 2 for tied positioning has previously been subjected 
to the customary preparations, in order to enable him to 
undergo the intervention. 

The means 2 of the tied positioning with respect to R 2 will 
not be described in detail since they can consist of any 50 
means (such as a retaining headset) normally used in the 
field of surgery or neurosurgery. The reference frame R 2 can 
arbitrarily be defined as a tri-rectangular reference trihedron 
Lied to the operating table TO, as represented in FIG, 1. 

Means 3 of marking, with respect to the second reference 55 
frame R 2 , the coordinates, denoted X2, Y2, Z2, of arbitrary 
points, and in particular of a certain number of base points 
of the reference structure SR are furthermore provided. 

These base points constituting the reference structure SR 
can consist of certain notable points and/or of marks fixed to 60 
the patient, at positions selected by the surgeon and in 
particular at these notable points. 

The system of the invention further comprises computing 
means 4 receiving means 3 of marking the coordinates X2, 
X2, 72, 65 

The computing means 4, as will be seen in detail later, are 
designed to elaborate optimal tools for reference frame 
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transfer using on the one hand the coordinates in R 2 , 
measured by the probe 3, of a plurality of base points of the 
structure SR, and on the other hand the coordinates in R lf 
determined by graphical tools of the computer M01 
(pointing by mouse, etc), of the images of the corresponding 
base points in the representation RSR, so as to secure the 
best possible correlation between the information modeled 
in the computer equipment and the corresponding real-world 
information. 

There is furthermore provision for reference frame trans- 
fer means 11 designed to use the tools thus elaborated and 
to secure this correlation in real lime. 

Moreover, means 40 are provided, as will be seen in detail 
later, lbr determining or modeling a reference origin of 
intervention ORI and a direction of intervention A. 

With the aid of the means 11, the modeled direction of 
intervention A can, at least prior to the intervention and at the 
start of the intervention, be materialized through an optical 
sighting system available to the surgeon, it being possible to 
steer this sighting system positionally with respect to the 
second reference frame R 2 . 

The sighting system will be described later. 

The system of the present invention finally comprises 
means 5 of intervention comprising an active member, 
denoted 50, whose position is specified with respect to the 
second reference frame R 2 . The active member can consist 
of the various tools used in surgical intervention. For 
example, in the case of an intercranial neurosurgical 
intervention, the active member could be a trephining tool, 
a needle, a laser or radioscope emission head, or an endo- 
scopic viewing system. 

According to an advantageous characteristic of the 
invention, by virtue of the reference frame transfer means 
11, the position of the active member can be controlled 
dynamically on the basis of the prior modeling of the origin 
of intervention ORI and of the direction of intervention A. 

The means 1 of dynamic display by three-dimensional 
imaging of the representations RSNH and RSR comprise a 
file 10 of two-dimensional image data. The file 10 consists 
for example of digitized data from tomographic sections, 
from radiographs, from maps of the patient's head, and 
contained in an appropriate mass memory. 

The successive tomographic sections can be produced 
prior to the intervention in a conventional manner, after the 
reference structure SR has been put in place on the nonho- 
mogeneous structure SNH. 

According to an advantageous feature, the reference 
structure SR can consist of a plurality of marks or notable 
points which can be both sensed by the marker means 3 and 
detected on the two-dimensional images obtained. 

Of course, the abovementioned two-dimensional tomo- 
graphic sections can likewise be produced by any medical 
imaging means such as a nuclear magnetic resonance sys- 
tem. 

In a characteristic and well-known manner, each two- 
dimensional image corresponding to a tomographic scanner 
section corresponds to a structural slice thickness of about 2 
to 3 mm, the pixels or image elements in the plane of the 
tomographic section being obtained with a precision of the 
order of ±1 mm. It is therefore understood that the marks or 
points constituting the reference structure SR appear on the 
images with a positional uncertainty, and an important 
feature of the invention will consist in minimizing these 
uncertainties as will be described later. 

The system also comprises first means 110 for calculating 
and reconstructing three-dimensional images from the data 
from the file 10. 
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It also comprises a high -resolution screen 12 permitting determine the coordinates of the said marks in the reference 

the displaying of one or more three-dimensional or two- frame R 2 or in a reference frame tied to the latter, 

dimensional images constituting so many representations of It is important to note here that the general function of the 

the reference structure RSR and of the nonhomogencous base points of the reference structure is, on the one hand, to 

structure SNH. 5 be individually localizable on the reference structure, in 

Advantageously, the calculating means 110, the high- order to deduce from this the coordinates in R 2 , and on the 

resolution screen and the mass memory containing the file other hand, to be visualizable on the two-dimensional 

10 form part of a computer of the workstation type with images so as to be identified (by their coordinates in R 2 ) and 

conventional design and denoted MOl. included in the representation RSR on the screen. 

Preferably, the first calculating means 110 can consist of io It can therefore involve special marks affixed at arbitrary 

a CAD type program installed in the workstation MOl. points of the lateral surface of the structure SNH, or else at 

By way of non-limiting example, this program can be notable points of the latter, or else, when the notable points 

derived from the software marketed under the tradename can in themselves be localized with high precision both on 

"AUTOCAD" by the "AUTODESK" company in the the structure SNH and on the 2D sections, notable points 

United Slates of America. is totally devoid of marks. 

Such software makes it possible, from the various digi- In FIG. 2 a plurality of marks, denoted Ml to Mi, these 

tized two-dimensional images, to reconstruct threedimen- marks, in the case where the nonhomogeneous structure 

sional images constituting the representations of the struc- consists of the head of a patient, being localized for example 

tures RSR and RSNH in arbitrary orientations. between the eyebrows of the patient, on the latter's temples, 

Thus, as has furthermore been represented in FIG. 1, the 20 and at the apex of the skull at a notable point such as the 

calculating means 4 and 11 can consist of a third computer, frontal median point. 

denoted M02 in FIG. 1. More generally, for a substantially ovoid volume consti- 

The first and second computers MOl and M02 are tuting the nonhomogeneous structure, there is advanta- 

interconnected by a conventional digital link (bus or geously provision for four base points at least on the outer 

network). 25 surface of the volume. 

As a variant, the computers MOl and M02 can be Thus, as has been represented in FIG. 3, the four marks 

replaced by a single workstation. Ml to M4 of the reference structure are distributed so as 

The marker means 3 consist of a three-dimensional probe preferably to define a more or less symmetric tetrahedron, 

equipped with a tactile tip 30. The symmetry of the tetrahedron, represented in FIG. 3, is 

This type of three-dimensional probe, known per se and 30 materialized by the vertical symmetry plane PV and the 

not described in detail, consists of a plurality of hinged arms, horizontal symmetry plane PH. 

marked in terms of position with respect to a base integral According to an advantageous characteristic, as will be 

with the operating table TO. It makes it possible to ascertain seen later, the means of elaborating the reference frame 

the coordinates of the tactile tip 30 with respect to the origin transfer tools are designed to select three points of the 

0 2 of the reference frame R 2 with a precision better than 1 35 tetrahedron which will define the "best plane" for the 

mm. reference frame transfer. 

The probe is for example equipped with resolvers deliv- Also, the presence of four or more points enables the 

ering signals representing the instantaneous position of the additional point(s) to validate a specified selection, 

abovernentioned tactile tip 30. The resolvers are themselves More precisely, the presence of a minimum of four base 

connected to the circuits for digital/analog conversion and 40 points on the reference structure makes it possible to search 

sampling of the values representing these signals, these for the minimum distortion between the points captured on 

sampling circuits being interconnected in conventional man- the patient by the marker means consisting for example of 

ner to the second computer M02 in order to supply it with the three-dimensional probe and the images of these points 

the coordinates X2, X2, 72 of the tactile tip 30. on the representation by three-dimensional imaging, the 

As a variant or additionally, and as represented 45 coordinates of which are calculated during processing. The 

diagramraatically, the marker means 3 can comprise a set of best plane of the tetrahedron described earlier, that is to say 

video cameras 31 and 32 (or else infrared cameras) enabling the plane for which the uncertainty in the coordinates of the 

pictures to be taken of the structures SNH and SR. points between the points actually captured by the three- 

The set of cameras can act as a stereoscopic system dimensional probe and the points of the representation of the 

permitting the positional plotting of the base points of the 50 reference structure RSR, is minimal, then becomes the 

reference structure SR, or of other points of the nonhomo- reference plane for the reference frame transfer. Thus, the 

geneous structure SNH, with respect to the second reference best correlation will be established between a modeled 

frame R 2 . The positional plotting can be done for example direction of intervention and a modeled origin of 

by appending a laser beam emission system making it intervention, on the one hand, and the action of the member 

possible to illuminate successively the points whose coor- 55 SO. Preferably, the origin of intervention will be placed at the 

dinates are sought, appropriate software making it possible center of the region in which the intervention is to be carried 

to then determine the position of these points one by one out, that is to say a tumor observed or treated for example, 

with respect to R 2 . This software will not be described since Furthermore, it will be possible to take the noted residual 

it can consist of position and shape recognition software uncertainty into account in order to effect the representation 

normally available on the market, 60 of the model and of the tools on the dynamic display means. 

According to another variant, the marker means 3 can A more detailed description of the means of intervention 

comprise a telemetry system. 5 will now be given in connection with FIG. 4. 

In this case, the marks of the structure SR can consist of Preferably, the means of intervention 5 comprise a car- 
small radiotransmitters implanted for example on the rel- riage 52 which is translationally mobile along the operating 
evant points of the patient's head and designed to be visible 65 table TO, for example on a rack, denoted 54, whilst being 
on the two-dimensional images, appropriate electromagnetic driven by a motor, not shown, itself controlled by the 
or optical sensors (not shown) being provided in order to computer M02 for example, via an appropriate link. This 
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movement system will not be described in detail since it 
corresponds to a conventional movement system available 
on the market. As a variant, the carriage 52 can be mobile 
over a distinct path separated from the operating tabic TO, 
or immobile with respect to the operating table and then 5 
constitute a support. 

The support carriage 52 comprises in the first place a 
sighting member OV, constituting the above-mentioned 
sighting system, which can consist of a binocular telescope. 

The sighting member OV enables the surgeon, prior to the 10 
actual intervention, or during the latter, to sight the pre- 
sumed position of the region in which the intervention is to 
be carried out. 

Furthermore, and in a non-limiting manner, with the 
sighting member OV can be associated a helium-neon laser 15 
emission system, denoted EL, making it possible to secure 
the aiming of a fine positioning or sighting laser beam on the 
structure SNH and in particular, as will be seen in detail 
later, to indicate to the surgeon the position of an entry point 
PE prior to the intervention, to enable the latter to open the 20 
skull at the appropriate location, and likewise to indicate to 
him what the direction of intervention will be. Additionally, 
the illuminating of the relevant point of the nonhomoge- 
neous structure or at the very least the lateral surface of the 
latter enables the video cameras 31 and 32 to carry out, if 25 
necessary, a positional plotting. 

Preferably, a system for measuring position by telemetry 
53 is provided to secure the precise measurement of the 
position of the support carriage 52 of the sighting member 
OV and of the laser emission system EL. During the 30 
operation, and in order to secure the intervention, the 
carriage 52 can be moved along the rack 54, the position of 
the carriage 52 being measured very precisely by means of 
the system 53. The telemetry system 53 is interconnected 
with the microcomputer M02 by an appropriate link. 35 

The means of intervention 5 can advantageously consist 
of a guide arm 51 for the active member 50. 

The guide arm 51 can advantageously consist of several 
hinged segments, each hinge being equipped with motors 
and resolvers making it possible to secure control of move- 40 
ment of the end of the support arm and the positional 
plotting of this same end and therefore of the active member 
50 according to six degrees of freedom with respect to the 
carriage 52. The six degrees of freedom comprise, of course, 
three translational degrees of freedom with respect to a 45 
reference frame tied to the carriage 52 and three rotational 
degrees of freedom along these same axes. 

Thus, the support arm 51 and the member 50 are marked 
in terms of instantaneous position with respect to the second 
reference frame R 2 , on the one hand by way of the positional 50 
plot of the mobile carriage 52 and, on the other hand, by way 
of the resolvers associated with each hinge of the support 
arm 51. 

In the case of an intracranial neurosurgical surgical 
intervention, the active member 50 can be removed and can 55 
consist of a trephining tool, a needle or radioactive or 
chemical implant, a laser or radioisotope emission head or 
an endoscopic viewing system. These various members will 
not be described since they correspond to instruments nor- 
mally used in neurosurgery. 60 

The materializing of the modeled direction of intervention 
can be effective by means of the laser emitter EL. This 
sighting being performed, the guide arm 51 can then be 
brought manually or in steered manner into superposition 
with the direction of intervention A. 65 

In the case of manual positioning, the resolvers associated 
with the sighting member OV and the laser emitter EL, if 
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appropriate, make it possible to record the path of the 
sighting direction, constituting in particular the actual direc- 
tion of intervention, on the representation of the nonhomo- 
geneous structure in the dynamic display means 1. 

Furthermore, as will be described later and in preferential 
manner, the intervening surgeon will be able firstly to define 
a simulated intervention path and steer thereto the move- 
ments of the active member 50 in the nonhomogeneous 
structure in order effectively to secure all or part of the 
intervention. 

In this case, the progress of the intervention tool 50 is then 
steered directly to the simulated path (data ORI, A) by 
involving the reference frame transfer means 11 in order to 
express the path in the reference frame R 2 , 

A more detailed description of the implementation of the 
operational mode of the system of the invention will now be 
described in connection with FIGS, So and Sb. 

According to FIG. 5a, the first step consists in obtaining 
and organizing in memory the two-dimensional image data 
(step 100). Firstly, the nonhomogeneous structure SNH is 
prepared. In the case of a neurosurgical intervention for 
example, this means that the patient's head can be equipped 
with marks constituting the base points of the reference 
structure SR. These marks can be produced by means of 
points consisting of a dye partially absorbing the X-rays, 
such as a radiopaque dye. 

The abovementioned marks are implanted by the surgeon 
on the patient's head at notable points of the latter [sic], and 
images can then be taken of the nonhomogeneous structure 
SNH by tomography for example, by means of an apparatus 
of the X-ray scanner type. 

This operation will not be described in detail since it 
corresponds to conventional operations in the field of medi- 
cal imaging. 

The two-dimensional image data obtained are then con- 
stituted as digitized data in the file 10, these data being 
themselves marked with respect to the reference frame K t 
and making it possible, on demand, to restore the two- 
dimensional images onto the dynamic display means 1, 
these images representing superimposed sections of the 
nonhomogeneous structure SNH. 

From the digitized image data available to the surgeon, 
the latter then proceeds, as indicated at 101 in FIG. 5a, to 
select the structures of interest of the abovementioned 
images. 

The purpose of this step is to facilitate the work of the 
surgeon by forming three-dimensional images which con- 
tain only the contours of the elements of the structure which 
are essential for geometrical definition and real-time moni- 
toring of the progress of the intervention. 

In the case where the nonhomogeneous structure SNH 
consists of the head of a patient, an analysis of the two- 
dimensional image data makes it possible, from values of 
optical density of the corresponding image-points, straight- 
away to extract the contours of the skull, to check the 
distance scales, etc. 

Preferably, the abovementioned operations are performed 
on a rectangle of interest for a given two-dimensional image, 
this making it possible, by moving the rectangle of interest, 
to cover the whole of the image. 

The above analysis is performed by means of suitable 
software which thus makes it possible to extract and vec- 
torize the contours of the structures which will be modeled 
in the representations RSNH and RSR. 

The structures modeled in the case of a neurosurgical 
intervention are for example the skull, the cerebral 
ventricles, the tumor to be observed or treated, the falx 
cerebri, and the various functional regions. 
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According to a feature of the interactive system of the 
invention, the surgeon may have available a digitizing table 
or other graphics peripheral making it possible, for each 
displayed two-dimensional image, to rectify or complete the 
definition of the contour of a particular region of interest. 

It will be noted finally that by superimposing the extracted 
contours on the displayed two-dimensional image, the sur- 
geon will be able to validate the extractions carried out. 

The extracted contours are next processed by sampling 
points to obtain their coordinates in the reference frame R lt 
it being possible to constitute these coordinates as an ASCII 
type file. This involves step 102 for generating the three- 
dimensional data base. 

This step is followed by a step 103 of reconstructing the 
three-dimensional model. This step consists firstly, with the 
aid of the CAD type software, in carrying out on the basis 
of the contours of the structures of interest constituted as 
vectorized two-dimensional images an extrapolation 
between the various sectional planes. 

The abovementioned extrapolation is carried out prefer- 
ably by means of a "B-spline" type algorithm which seems 
best suited. This extrapolation transforms a discrete item of 
information, namely the successive sections obtained by 
means of the scanner analysis, into a continuous model 
permitting three-dimensional representation of the volume 
envelopes of the structures. 

It should be noted that the reconstruction of the volumes 
constituting the structures of interest introduces an approxi- 
mation related in particular to the spacing and non-zero 
thickness of the acquisition sections. An important charac- 
teristic of the invention, as explained in detail elsewhere, is 
on the one hand to minimize the resulting uncertainties in the 
patient-model correlation, and on the other hand to take into 
account the residual uncertainties. 

The CAD type software used possesses standard functions 
which enable the model to be manipulated in space by 
displaying it from different viewpoints through just a crite- 
rion defined by the surgeon (step 104). 

The software can also reconstruct sectional representation 
planes of the nonhomogeneous structure which differ from 
the planes of the images from the file 10, this making it 
possible in particular to develop knowledge enhancing the 
data for the representation by building up a neuro- 
anatomical map. 

The surgeon can next (step 105) determine a model of 
intervention strategy taking into account the modeled struc- 
tures of interest, by evaluating the distance and angle ratios 
on the two-and three-dimensional representations displayed. 

This intervention strategy will consist, in actual fact, on 
the one hand in localizing the tumor and in associating 
therewith a "target point", which will subsequently be able 
to substitute for the origin common to all the objects (real 
and images) treated by the system, and on the other hand in 
determining a simulated intervention path respecting to the 
maximum the integrity of the structures of interest. This step 
can be carried out "in the office", involving only the work- 
station. 

Once this operation is performed and prior to the 
intervention, the following phase consists in implementing 
the steps required to establish as exact as possible a corre- 
lation between the structure SNI-I (real world) and the 
representation RSNH (computer world). This involves steps 
106 to 109 of FIG. 5b. 

Firstly, as represented in FIG. Sb at step 107, marking of 
the base points of the reference structure SR with respect to 
the second reference frame is carried out with the aid of the 
marker means 3, by delivering to the system the coordinates 
X2, Y2, Z2 of the said base points. 
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The following step 106 consists in identifying on the 
representations RSNH and RSR displayed on the screen the 
images of the base points which have just been marked. 
More precisely, with the aid of appropriate graphics 

5 peripherals, these representations (images) of the base points 
are selected one by one, the workstation supplying on each 
occasion (in this instance to the computer M02) the coor- 
dinates of these points represented in the reference frame R 3 , 
Thus the computer M02 has available a first set of 

10 three-dimensional coordinates representing the position of 
the base points in R 2 , and a second set of three-dimensional 
coordinates representing the position of the representations 
of the base points in R 3 . 

According to an essential feature of the invention, these 

15 data will be used to elaborate at 108, 109, tools for reference 
frame transfer (from R 3 to R 2 and vice versa) by calling 
upon an intermediate reference frame determined from the 
base points and constituting an intermediate reference frame 
specific to the reconstructed model. 

20 More precisely, the intermediate reference frame is con- 
structed from three base points selected so that, in this 
reference frame, the coordinates of the other base points 
after transfer from R 2 and the coordinates of the represen- 
tations of these other base points after transfer from R 3 are 

25 expressed with the greatest consistency and minimum dis- 
tortion. 

When the step of elaborating the reference frame transfer 
tools is concluded, these tools can be used by he system to 
secure optimal coupling between the real world and the 

30 computer world (step 1110). 

Furthermore, according to a subsidiary feature of the 
present invention, the system can create on the display 
means a representation of the nonhomogeneous structure 
and of the intervention member which takes account of the 

35 deviations and distortions remaining after the "best" refer- 
ence frame transfer tools have been selected (residual 
uncertainties). More precisely, from these deviations can be 
deduced by the calculating means a standard error likely to 
appear in the mutual positioning between the representation 

40 of the nonhomogeneous structure and the representation of 
elements (tools, sighting axes, etc.) referenced on R 2 when 
using the reference frame transfer tools. This residual 
uncertainty, which may in practice be given substance 
through an error matrix, can be used for example to repre- 

45 sent certain contours (tool, structures of interest to be 
avoided during the intervention, etc.) with dimensions larger 
than those which would normally be represented starting 
from the three-dimensional data base or with the aid of 
coordinates marked in R 2 , the said larger dimensions being 

50 deduced from the "normal" dimensions by involving the 
error matrix. For example, if the member were represented 
normally, in transverse section, by a circle of diameter Dl, 
a circle of diameter D2>D1 can be represented in substance, 
with the difference D2-D1 deduced from the standard error 

55 value. In this way, when a direction of intervention will be 
selected making it possible to avoid traversing certain struc- 
tures of interest, the taking into account of an "enlarged" size 
of the intervention tool will eradicate any risk of the 
member, because of the abovementioned errors, accidently 

60 traversing these structures. 

Back at step 105, and as will be seen in more detail with 
reference to FIGS. 9a and 9b, the reference origin of 
intervention ORI and the direction of intervention A, that is 
to say the simulated intervention path, can be determined 

65 according to various procedures. 

According to a first procedure, the trajectory can be 
defined from two points, namely an entry point PE (FIG. 3) 
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and a target point, that is to say substantially the center of the 
structure of interest consisting of the tumor to be observed 
or treated. Initially, these two points are localized on the 
model represented on the screen. 

According to a second methodology, the trajectory can be 
determined from the abovementioned target point and from 
a direction which takes account of the types of structures of 
interest and of their positions with a view to optimally 
respecting their integrity. 

After the abovementioned step 108, the surgeon can at 
step 1110 perform the actual intervention. 

The intervention can advantageously be performed by 
steering Lhe tool or active member over the simulated 
intervention path, determined in step 1110. 

As a variant, given that the support arm 51 for the active 
member, equipped with its resolvers, continuously delivers 
the coordinates in R 2 of the said active member to the 
system, it is also possible to perform the operation manually 
or semi-manually, by monitoring on the screen the position 
and motions of a representation of the tool and by comparing 
them with the simulated, displayed intervention path. 

It will furthermore be noted that the modeled direction of 
. intervention can be materialized with the aid of the laser 
beam described earlier, the positioning of the latter (with 
respect to R 2 ) being likewise carried out by virtue of the 
reference frame transfer tools. 

Certain functional features of the system of the invention 
will now be described in further detail with reference to 
FIGS. 6, 7, 8, 9a and 9b. 

The module for elaborating the reference frame transfer 
tools (steps 108, 109 of FIG. Sb) will firstly be described 
with reference to FIG, 6, 

This module comprises a first sub-module 1001 for 
acquiring three points A, B, C, the images of the base points 
of SR on the representation RSNH (the coordinates of these 
points being expressed in the computer reference frame R I ), 
by successive selections of these points on the representa- 
tion. To this effect, the surgeon is led, by means of a graphics 
interface such as a "mouse" to point successively at the three 
selected points A, B, C. 

The module for preparing the transfer tools also com- 
prises a second sub-module, denoted 1002, for creating a 
unit three-dimensional orthogonal matrix M, this matrix 
being characteristic of a right-handed orthonormal basis 

represented by three unit vectors T, j , k, which define an 
intermediate reference frame tied to R 1 . 

The unit vectors T, j and k are given by the relations: 



k=\ BA ABC /|| BA ABC \ 



i - / A k 

where || || designates the norm of the relevant vector. 

In the above relations, the sign "A" designates the vector 
product of the relevant vectors. 

Similarly, the module for preparing the transfer tools 
comprises a third sub-module, denoted 1003, for acquiring 
three base points D, £, F, of the structure SR, these three 
points being those whose images on the model are the points 
A, B, C respectively. For this purpose, the surgeon, for 
example by means of the tactile tip 30, successively senses 
these three points to obtain their coordinates in R 2 . 
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The sub-module 1003 is itself followed, as represented in 
FIG. 6, by a fourth sub-module 1004 for creating a unit 
three-dimensional orthogonal matrix N, characteristic of a 
right-handed orthonormal basis comprising three unit vec- 
tors i j ' k* and which is tied to the second reference 
frame R 2 owing to the fact that the nonhomogeneous struc- 
ture SNH is positionally tied with respect to this reference 
frame. 



The three unit vectors 
relations: 



j'-DE(\\DE\\ 



EDAEF l/||£DA£F|| 



i j k 1 are defined by the 



/ ' = / ' A k* 

As indicated above, to the extent that the base points of 
the reference structure can be marked in R 2 with high 
precision, so their representation in the computer base R a is 
marked with a certain margin of error given on the one hand 
the non-zero thickness (typically from 2 to 3 mm) of the 
slices represented by the two-dimensional images from the 
file 10, and on the other hand (in general to a lesser extent) 
the definition of each image element or pixel of a section. 

According to the invention, once a pair of transfer matri- 
ces M, N has been elaborated with selected points A, B, C, 
D, E, F, it is sought to validate this selection by using one or 
more additional base points; more precisely, for the or each 
additional base point, this point is marked in R 2 with the aid 
of the probe 30, the representation of this point is marked in 
Rj after selection on the screen, and then the matrices N and 
M are applied respectively to the coordinates obtained, in 

order to obtain their expressions in the bases (T l , "j\ ^ ') and 

( i , j , k) respectively. If these expressions are in good 
agreement, these two bases can be regarded as a single 
intermediate reference frame, this securing the exact as 
possible mathematical coupling between the computer ref- 
erence frame R 1 tied to the model and the "real" reference 
frame R 2 tied to the patient. 

In practice, the module for elaborating the reference frame 
transfer tools can be designed to perform steps 1001 to 1004 
in succession on basic triples which differ on each occasion 
(for example, if four base points have been defined associ- 
ated with four representations in RSR, there are four pos- 
sible triples), in order to perform the validation step 1005 for 
each of these selections and finally in order to choose the 
triple for which the best validation is obtained, that is to say 
for which the deviation between the abovementioned 
expressions is smallest. This triple defines the "best plane" 
mentioned elsewhere in the description, and results in the 
"best" transfer matrices M and N. 

As a variant, it will be possible for the selection of the best 
plane to be made at least in part by the surgeon by virtue of 
his experience. 

It should be noted that the reference frame transfer will 
only be concluded by supplementing the matrix calculation 
with the matrices M, N with a transfer of origin, so as to 
create a new common origin for example at the center of the 
tumor to be observed or treated (point ORI). This transfer of 
origin is effected simply by appropriate subtraction of vec- 
tors on the one hand on the coordinates in R 2 , and on the 
other hand on the coordinates in R 2 . These vectors to be 
subtracted are determined after localization of the center of 
the tumor on the representation. 
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Furthermore, Lhe means described above for establishing From a simulated intervention path, for example on the 

the coupling between the patient's world and the model's representation RSNH, or from any other straight line 

world can also be used to couple to the model's world that selected by the surgeon, the invention furthermore enables 

of map data, also stored in the workstation and expressed in the mociel t0 be repr esented on the screen from a viewpoint 

a different reference frame denoted R 3 . In this case, since 5 corrcsponding t0 this straight Hne , Xhus the third transfer 

these data contain no specific visible mark, the earlier . . 4 , . a j nL 

described elaboration of matrices is performed by subslitut- subroutine composes, as represented in FIGS. 9a and 9 b, a 

ing for these marks the positions of notable points of the hrst module 301 tor v ^ahzing the representation m a 

patient's head. These may be temporal points, the frontal direction given by two points and a second module 302 for 

median point, the apex of the skull, the center of gravity of visualizing the representation in a direction given by an 

the orbits of the eyes, etc. angle of elevation and an angle of azimuth, 

The corresponding points of the model can be obtained The first module 301 for visualizing the representation in 

either by selection by mouse or graphics tablet on the model, a direction given by two points comprises a first sub-module 

or by sensing on the patient himself and then using the denoted 3010 permitting acquisition of the two relevant 

transfer matrices. _ points which will define the selected direction. The coordi- 

The above step of elaborating the reference frame transfer * nateg of thege ^ are reforence ffame 

tools, conducted m practice by the calculating means 4, . . , . . , , . , . , 

makes it possible subsequently to implement the reference these l )0ints havin & either been acc J uired Previously on the 

frame transfer means (FIGS. 7 and 8). nonhomogeneous structure SNH for example by means of 

With reference to FIG. 7, the first transfer sub-module 201 the tactile tip 30 and then subjected to the reference frame 

comprises a procedure denoted 2010 for aquiring the coor- 20 transfer, or chosen directly on the representation by means 

dinates XM, YM, ZM, expressed in R a , of the point to be of the graphics interface of the "mouse" type, 

transferred, by selecting on the representation. The first sub-module 3010 is followed by a second 

1 ne procedure 2010 followed by a procedure 2011 for sub-module denoted 3011 permitting the creation of a unit, 

calculating the coordinates XP, YP, ZP (expressed in R 2 ) of orthogonal three-dimensional matrix V characteristic of a 

the corresponding real point on the patient through the 05 .... , , , . . -t» 7* A , 

transformation* right-handed orthonormal basis 1 \ j k " the unit vectors 

{XP, YP, ZP}=M*N~ 3 *{XM, YM, ZM} where M * N" 1 T B , j\ k*\ being determined through the relations: 
represents the product of the matrix M and the inverse 
matrix N. 

The procedure 2011 is followed by a processing proce- 30 k n "AB/[\AB\\\ 

dure 2012 utilizing the calculated coordinates XP, YP, ZP, 7 M *7 M » o-?' « 0;\\ f j| » i; 

for example to indicate the corresponding point on the _ 

surface of the structure SNH by means of the laser emission = A; M 
system EL, or again to secure ihe intervention at the relevant 

point with coordinates XP, YP, ZP (by steering the active 35 where "A" represents the vector product and symbolizes 

member), the scalar product. 

^f e [ Sely ' " f d6 , r t0 T? re * traDSfei : fr °^ S i^x t0 The sub-module 3011 is followed by a routine 3012 

RSNH, the second sub-module 202 composes (FIG. H) a maW it possib]e t0 secure for all the points of the entjlies 

procedure denoted 2020 for acquiring on the structure SNH (structures of ^rzst) of the three-dimensional data base of 

the coordinates XP,YP,ZP (expressed in R 2 ) of a point to be 40 coor dinates XW, YW, ZW in R, a conversion into the 

transferred. ^ 

These coordinates can be obtained by means of the tactile orthonormal basis ( i j k ") by the relation: 

tip 30 for example. The procedure 2020 is followed by a {XV, YV, ZV}=V*{XW, YW, ZW} 

procedure 2021 for calculating the corresponding coordi- The subroutine 3013 is' then' followed by a subroutine 

nates XM, YM, ZM in R a through the transformation: 45 3014 for displaying the plane i\j\ the subroutines 3013 and 

{XM, YM, ZM}=N*M~ X * {XP, YP, ZP} 3014 being called up for all the points, as symbolized by the 

A procedure 2022 next makes it possible to effect the arrow returning to block 3012 in FIG. 9a. 

displaying of the point with coordinates XM, YM, ZM on When all the points have been processed, an output 

the model or again of a straight line or of a plane passing module 3015 permits return to a general module, which will 

through this point and furthermore meeting other criteria. 50 be described later in the description. It is understood that this 

It will be noted here that the two sub-modules 201, 202 module enables two-dimensional images to be reconstructed 

can used [sic] by the surgeon at any moment for the purpose in planes perpendicular to the direction defined by A and B. 

of checking the valid nature of the transfer tools; in In the same way, the second module 302 (FIG. 9b) for 

particular, it is possible to check at any time thai a real base visualizing the representation from a viewpoint given by an 

point, with coordinates known both in R 2 and R a (for 55 angle of elevation and an angle of azimuth comprises a first 

example a base point of SR or an arbitrary notable point of sub-module 3020 for acquiring the two angles in the repre- 

the structure SNH visible on the images), correctly relocates sentation frame of reference. 

with respect to its image after transferring the coordinates in The selection of the angles of elevation and of azimuth 

step 2011. can be made by selecting from a predefined data base or by 

In the event of an excessive difference, a new step of 60 moving software cursers associated with each view or else 

elaboration of the transfer tools is performed. by modification relative to a current direction, such as the 

Furthermore, the sub-modules 201, 202 can be designed modeled direction of intervention. The sub-module 3020 is 

to also integrate the taking into account of the residual itself followed by a second sub-module 3021 for creating a 

uncertainty, as spoken of above, so as for example to unit orthoganal three-dimensional matrix W characteristic of 

represent on the screen a point sensed not in a pointwise 65 . . , . , , . . . „ -t> 

manner, but in the form for example of a circle or a sphere ^ nght-handed orthonormal bas.s of unit vectors i "', j "', 

representing the said uncertainty. k'". They are defined by the relations: 
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7"' • ~k » 0; 

A* "' * r = sin(azimuth) 

/ • y = cos (elevation); 
/ * ,t *' = sin (elevation) 
7"' = A 7"' 

A routine 3022 is then called for all the points of the 
entities of the three-dimensional data base of coordinates 
XW, YW, ZW and enables a first sub-routine 3023 to be 
called permitting calculation of the coordinates of the rel- 
evant point in the right-handed orthonormal bases i 

X'" through the transformation: 

{XV, YV, ZV}-V*{XW, YW, ZW} 

The sub-routine 3023 is itself followed by a sub-routine 
3024 for displaying the plane i ,M , j ,H , the two sub-routines 
3023 and 3024 then being called up for each point as 
symbolized by the return via the arrow to the block 3022 for 
calling the abovementioned routine. When all the points 
have been processed, an output sub-module 3025 permits a 
return to the general menu. 

Of course, all of the programs, sub-routines, modules, 
sub-modules and routines destroyed earlier are managed by 
a general "menu" type program so as to permit interactive 
driving of the system by screen dialogue with the interven- 
ing surgeon by specific screen pages. 

A more specific description of a general flow diagram 
illustrating this general program will now be given in 
connection with FIGS. 10a and 10b, 

r lhus, in FIG. 10a has been represented in succession a 
screen page 4000 relating to the loading of data from the 
digitized file 10, followed by a screen page 4001 making it 
possible to secure the parameterizing of the grey scales of 
the display on the dynamic display means 1 and to calibrate 
the image, for example. 

The screen page 4001 is followed by a screen page 4002 
making it possible to effect the generation of a global view 
and then a step or screen page 4003 makes it possible to 
effect an automatic distribution of the sections on the screen 
of the workstation. 

A screen page 4004 makes it possible to effect a manual 
selection of sections and then a screen page 4005 makes it 
possible to effect the selection of the strategy (search for the 
entry points and for the possible directions of intervention, 
first localizing of the target (tumor . . . ) to be treated . . . ), 
as defined earlier, and to select the position and horizontal, 
sagittal and frontal distribution of the sections. 

A screen page 4006 also makes it possible to effect a 
display of the settings of a possible stereotaxic frame. 

It will be recalled that the reference structure SR advan- 
tageously replaces the stereotaxic frame formerly used to 
effect the marking of position inside the patient's skull. 

There may furthermore be provided a screen-page 4007 
for choosing strategic sections by three-dimensional 
viewing, on selection by the surgeon, and then at 4008 the 
aligning of the references of the peripherals (tool, sighting 
members, etc., with the aid of the probe 30. 

A screen page 4009 is also provided to effect the search 
for the base points on the patient with the aid of the said 
probe, following which the steps of construction of the 
reference frame transfer tools and of actual reference frame 
transfer are performed, preferably in a user-transparent 
manner. 
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Another screen page 4010 is then provided, so as to effect 
the localizing of the target on the representation (for 
example a tumor to be observed or treated in the case of a 
neurosurgical intervention) in order subsequently to deter- 

5 mine a simulated intervention path. 

Then a new screen page 4011 makes it possible to effect 
the setting of the guides for the tool on the basis of this 
simulated path before opening up the skin and bone flaps on 
the patient's skull 

Then a new localizing step 4012 makes it possible to 
check whether the position of the guides corresponds cor- 
rectly to the simulated intervention path. 

The screen page 4012 is followed by a so-called inter- 
vention screen page, the intervention being performed in 
accordance with step 1110 of FIG. 5b. 

15 A more detailed description of the interactive dialogue 
between the surgeon and the system during a surgical, and 
in particular a neurosurgical, intervention will follow with 
reference to FIG. 10c and to all of the preceding description. 
The steps of FIG. 10c are also integrated in the general 

20 program mentioned earlier; there are undertaken in succes- 
sion a first phase I (preparation of the intervention), then a 
second phase II, (prior to the actual intervention, the patient 
is placed in a condition for intervention, the reference 
structure SR being tied to the second reference frame R^, 

25 then a third phase III (intervention) and finally a post- 
intervention phase IV. 

With a view to preparing the intervention, the system 
requests the surgeon (step 5000) to choose the elementary 
structures of interest (for example bones of the skull, 

3Q ventricles, vascular regions, the tumor to be explored or 
treated, and the images of the marks constituting in the first 
reference frame the representation RSR). 

The choice of the elementary structures of interest is made 
on the display of the tomographic images, for example, 
called up from the digitized file 10. 

35 The system next performs, at step 5001, a modeling of the 
structures of interest, as described earlier. Then, the nonho- 
mogencous structure having been thus constituted as a 
three-dimensional model RSNH displayed on the screen, the 
intervening surgeon is then led to perform a simulation by 

40 three-dimensional imaging, at step 5002, with a view to 
defining the intervention path of the tool 50. 

During phase II the patient being placed in a condition for 
intervention and his head and the reference structure SR 
being tied to the second reference frame R 2 , the surgeon 

45 performs at step 5003 a search for the position of the marks 
Ml to M4 constituting base points of the reference structure 
in the second reference frame R 2 , and then during a step 
5004, performs a search for the position of the sighting 
systems, visualizing member OV, or of the tools and inter- 
so vention instruments 50, still in the second reference frame 
R 2 , so as, if appropriate, to align these implements with 
respect to R 2 . 

The system then performs the validation of the 
intervention/patient spaces and representation by three- 

55 dimensional imaging in order to determine next the common 
origin of intervention ORL In other words, the matrix 
reference frame transfer described above is supplemented 
with the necessary origin translations (origins 01 and 02 
aligned on ORI). 

60 This operation is performed as described earlier. 

Phase III corresponds to the intervention, during which 
the system effects at step 5006 a permanent coupling in real 
time between the direction of aim of the active member 50, 
and/or of the direction of aim of the sighting member OV 

65 (and if appropriate of the laser beam), with the direction of 
aim (of observation) simulated by three-dimensional imag- 
ing on the display means 1, and vice versa. 
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In the following step 5007, the coupling is effected of the 
movements and motions of the intervention instrument with 
their movements simulated by three-dimensional imaging, 
with automatic or manual conduct of the intervention. 

As noted at 5008, the surgeon can be supplied with a 5 
permanent display of the original two-dimensional sectional 
images in planes specified with respect to the origin ORI and 
to the direction of intervention. Such a display enables the 
surgeon at any time to follow the progress of the intervention 
in real time and to be assured that the intervention is 10 
proceeding in accordance with the simulated intervention. In 
phase IV which is executed after the intervention, the system 
effects a saving of the data acquired during the intervention, 
this saving making it possible subsequently to effect a 
comparison in real time or deferred in the event of succes- is 
sive interventions on the same patient. 

Furthermore, the saved data make it possible to effect a 
playback of the operations carried out with the option of 
detailing and supplementing the regions traversed by the 
active member 50. 20 

Thus, a particularly powerful interactive system for local 
intervention has been described. 

Thus, the system which is the subject of the present 
invention makes it possible to represent a model containing 
only the essential structures of the nonhomogeneous 25 
structure, this facilitating the work of preparation and of 
monitoring of the intervention by the surgeon. 

Moreover, the system, by virtue of the algorithms used 
and in particular by minimizing the distortion between the 
real base points and their images in the 2D sections or the 30 
maps, makes it possible to establish a two-way coupling 
between the real world and the computer world through 
which the transfer errors are minimized, making possible 
concrete exploitation of the imaging data in order to steer the 
intervention tool. 35 

To summarize, the system makes possible an ineractive 
[sic] medical usage not only to create a three-dimensional 
model of the nonhomogeneous structure but also to permit 
a marking in real time with respect to the internal structures 
and to guide the surgeon in the intervention phase, 40 

More generally, the invention makes it possible to end up 
with a coherent system in respect of: 

the two-dimensional imaging data (scanner sections, 
maps, etc) 

the three-dimensional data base; 45 
the data supplied by the marker means 3 in the reference 
frame R 2 ; 

the coordinate data for the sighting systems and interven- 
tion tools; 

the real world of the patient on the operating table. 

Accordingly, the options offered by the system are, in a 
non-limiting manner, the following: 

the tools and of [sic] their position can be represented on 
the screen; 55 

the position of a point on the screen can be materialized 
on the patient for example with the aid of the laser 
emission device EL; 

the orientation and the path of a tool such as a needle can 
be represented on the screen and materialized on the go 
patient optically (laser emission) or mechanically 
(positioning of the guide-arm in which the tool is 
guided in translation): 

an image of the patient, yielded for example by a system 
for taking pictures if appropriate in relief, can be 65 
superimposed on the three-dimensional representation 
modeled on the screen; thus, any change in the soft 
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external parts of the patient can be visualized as com- 
pared with the capture by the scanner; 

it being possible for the surgeon's field of view given by 
a sighting member (such as a surgical microscope) to be 
referenced with respect to R 2 , the direction of visual- 
ization of the model on the screen can be made identical 
to the real sight by the sighting member; 

finally, the three-dimensional images, normally displayed 
on the screen in the preceding description, may as a 
variant be introduced into the surgeon's microscope so 
as to obtain the superposition of the real image and the 
representation of the model. 

We claim: 

1. An interactive system for local intervention inside a 
region of a non-homogeneous structure to which is con- 
nected a reference structure containing a plurality of base 
points, the interactive system comprising: 

means for dynamically displaying a three-dimensional 
image of a representation of the non-homogeneous 
structure and of the reference structure connected to the 
non-homogeneous structure, wherein the three- 
dimensional image also includes a plurality of images 
of the plurality of base points; 

means for determining a set of coordinates of the plurality 
of images of the plurality of base points in a first 
reference frame; 

means for fixing a position of the non-homogeneous 
structure and of the reference structure with respect to 
a second reference frame; 

means for determining a set of coordinates of the plurality 
of base points in the second reference frame; 

means of intervention comprising an active member 
whose position is determined with respect to the second 
reference frame; 

means for generating a plurality of reference frame trans- 
lation tools for translating a plurality of reference 
frames from the first reference frame to the second 
reference frame and vice versa, based on the set of 
coordinates of the plurality of images of the plurality of 
base points in the first reference frame and of the set of 
coordinates of the plurality of base points in the second 
reference frame, in such a way as to reduce to a 
minimum at least one of a set of deviations between the 
set of coordinates of the plurality of images of the 
plurality of base points in the first reference frame and 
the set of coordinates of the base points, expressed in 
the first reference frame using the plurality of reference 
frame translation tools; 

means for defining, with respect to the first reference 
frame, a simulated origin of intervention and a simu- 
lated direction of intervention; and, 

means for transferring the plurality of reference frames 
using the plurality of reference frame translation tools 
to establish a bidirectional coupling between the simu- 
lated origin of intervention and the simulated direction 
of intervention and the position of the active member. 

2, The interactive system according to claim 1, wherein 
the plurality of reference frame translation tools comprise: 

means for creating a matrix (M) for transferring between 
the first reference frame and a first intermediate refer- 
ence frame based on a set of coordinates of a set of 
three images of a set of three base points of the 
reference structure; 

means for creating a matrix (N) for transferring between 
the second reference frame and a second intermediate 
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reference frame based on the set of coordinates of the 
set of three images of the set of three base points of the 
reference structure; and, 
means for validating matrix (M) and matrix (N) based on 
the set of three base points and the set of three images, 5 
such that at least one deviation between an expression 
for at least one additional base point in the second 
intermediate reference frame and an expression for at 
least one image point of the additional base point in the 
first intermediate reference frame is reduced to a mini- 10 
mum. 

3. The interactive system according to plurality of claim 
2, wherein the means for transferring the reference frames 
using the plurality of reference frame translation tools 
further comprises: 15 

a first transfer sub-module for transferring a set of 
rep rcsentation/non- homogeneous structure 
coordinates, and 

a second transfer sub-module for transferring a set of 0Q 
non-homogeneous structure/representation coordi- 
nates. 

4. The interactive system according to claim 3, wherein 
the first transfer sub-module comprises: 

means for acquiring a set of coordinates (XM, YM, ZM), 2 $ 
expressed in the first reference frame, of a point of the 
representation of the non-homogeneous structure to be 
transferred, by selection on the representation; 

means for calculating a set of corresponding coordinates 
(XP, YP, ZP), expressed in the second reference frame, 30 
on the non-homogeneous structure through a transfor- 
mation: 

{YP,YP, ZP}=M*N" 3 *{XM,YM,ZM} where M * NT 1 
represents a product of the matrix (M) and an inverse 
of the matrix (N), and 35 

means for processing, with the aid of the corresponding 
coordinates (YP, YP, ZP), to display a corresponding 
point on a surface of the non-homogeneous structure 
and to secure the intervention. 

5. The interactive system according to claim 3, wherein 40 
the second transfer sub-module comprises: 

means for acquiring a set of coordinates (XP, YP, ZP), 
expressed in the second reference frame, of a point of 
the non-homogeneous structure to be transferred; ^ 

means for calculating a set of corresponding coordinates 
(XM YM, ZM), expressed in the first reference frame, 
of the representation through a transformation: 

{YM, YM, ZM}=N*M" 1 *{XP,ZP,ZP} where N*M" 1 
represents the product of the matrix (N) and an inverse 50 
of the matrix (M); and, 

means for displaying the representation using the set of 
corresponding coordinates (YM, YM, ZM). 

6. The interactive system according to claim 1, wherein 
the means for generating the plurality of reference frame 55 
translation tools also generate, in association with the ref- 
erence frame translation tools, tools for taking into account 

a residual uncertainty which is based on the set of deviations 
between the set of coordinates of the plurality of images of 
the plurality of base points in the first reference frame and 60 
the set of coordinates of the base points, the tools for taking 
into account the residual uncertainty usable for displaying a 
set of contours in the representation whilst taking into 
account the residual uncertainties. 

7. The interactive system according to claim 1, wherein 65 
the means of dynamic displaying the three-dimensional 
image comprises: 
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a file containing digitized data from a set of two- 
dimensional images constituted by successive non- 
invasive tomographic sections of the non- 
homogeneous structure; 

means for calculating and reconstructing the three- 
dimensional image from the set of two-dimensional 
images; and 

a high-resolution display screen. 

8. The interactive system according to claim 7, wherein 
the means for calculating and reconstructing the three- 
dimensional image from the set of two-dimensional images 
comprises a program consisting of computer-aided design 
type software. 

9. The interactive system according to claim 1, wherein 
the means for determining the set of coordinates of the 
plurality of base points in the second reference frame 
comprises a three-dimensional probe equipped with a tactile 
tip for delivering a set of coordinates of the tactile tip in the 
said second reference frame. 

10. The interactive system according to claim 1, wherein 
the means for determining the set of coordinates of the 
plurality of base points is the second reference frame com- 
prises at least one of a set of optical sensors and a set of 
electromagnetic sensors. 

11. The interactive system according to claim 1, wherein 
a portion of the set of the plurality of base points of the 
reference structure comprises a plurality of marks positioned 
on a lateral surface of the non-homogeneous structure. 

12. The interactive system according to claim 11, wherein 
the plurality of marks are four in number and are distributed 
over the lateral surface so as to define a substantially 
symmetrical tetrahedron. 

13. The interactive system according to claim 1, wherein 
the means of intervention comprises: 

a guide arm to secure intervention in the region of the 
non-homogeneous structure, the guide arm having a 
position marked with respect to the second reference 
frame; and, 

an active intervention member whose position is marked 
with respect to the second reference frame. 

14. The interactive system according to claim 13, wherein 
the active intervention member is removable and selected 
from the group consisting of: 

tools for trephining; 
needles and implants; 

laser and radioisotope emission heads; and, sighting and 
viewing systems. 

15. The interactive system according to claim 1, wherein 
the means for transferring the plurality of reference frames 
establishes a coupling between a direction of visualization of 
the representation of the non-homogeneous structure on the 
display means and a direction of observation of the non- 
homogeneous structure and of the reference structure by the 
active intervention member. 

16. The interactive system according to claim 15, further 
comprising: 

a first module for visualizing a representation in a direc- 
tion given by two points; 

a second module for visualizing a representation in a 
direction given by an angle of elevation and an angle of 
azimuth. 

***** 



UNITED STATES PATENT AND TRADEMARK OFFICE 

CERTIFICATE OF CORRECTION 

PATENT NO.: 5,868,675 

DATED: February 9, 1999 

INVENTOR(S): Henrion et al. 

ft is aerified that error appears in the above-identified patent and that said Letters Patent is hereby 
corrected as shown below: 

In the Title, item [54], delete "NONHUMOGENEOUS STRUCTURE" and 
insert - NON-HOMOGENEOUS STRUCTURE --. 

At item [22], the PCT filing date, delete "May 10, 1990" and 
insert — October 5, 1990 --. 



Attest: 



Signed and Sealed this 
Eighth Day of May, 2001 



NICHOLAS P.GODICI 

Attesting Officer Acting Director nftlte United Slates Patent and Trademark Office 



Listing Of Claims 

1. An interactive system for local intervention inside a region of a non- 
homogeneous structure to which is connected a reference structure containing a 
plurality of base points, the interactive system comprising: 

means for dynamically displaying a three-dimensional image of a 
representation of the non-homogeneous structure and of the reference structure 
connected to the non-homogeneous structure, wherein the three-dimensional image 
also includes a plurality of images of the plurality of base points; 

means for determining a set of coordinates of the plurality of images of the 
plurality of base points in a first reference frame; 

means for fixing a position of the non-homogeneous structure and of the 
reference structure with respect to a second reference frame; 

means for determining a set of coordinates of the plurality of base points 
in the second reference frame; 

means of intervention comprising an active member whose position is 
determined with respect to the second reference frame; 

means for generating a plurality of reference frame translation tools for 
translating a plurality of reference frames from the first reference frame to the second 
reference frame and vice versa, based on the set of coordinates of the plurality of 
images of the plurality of base points in the first reference frame and of the set of 
coordinates of the plurality of base points in the second reference frame, in such a way 
as to reduce to a minimum at least one of a set of deviations between the set of 
coordinates of the plurality of images of the plurality of base points in the first reference 



frame and the set of coordinates of the base points, expressed in the first reference 
frame using the plurality of reference frame translation tools; 

means for defining, with respect to the first reference frame, a simulated 
origin of intervention and a simulated direction of intervention; and, 

means for transferring the plurality of reference frames using the plurality 
of reference frame translation tools to establish a bidirectional coupling between the 
simulated origin of intervention and the simulated direction of intervention and the 
position of the active member. 

2. The interactive system according to claim 1, wherein the plurality of 
reference frame translation tools comprise: 

means for creating a matrix (M) for transferring between the first reference 
frame and a first intermediate reference frame based on a set of coordinates of a set of 
three images of a set of three base points of the reference structure; 

means for creating a matrix (N) for transferring between the second 
reference frame and a second intermediate reference frame based on the set of 
coordinates of the set of three images of the set of three base points of the reference 
structure; and, 

means for validating matrix (M) and matrix (N) based on the set of three 
base points and the set of three images, such that at least one deviation between an 
expression for at least one additional base point in the second intermediate reference 
frame and an expression for at least one image point of the additional base point in the 
first intermediate reference frame is reduced to a minimum. 



Serial No. 09/784,829 



Page 2 of 28 



3. The interactive system according to plurality of claim 2, wherein the 
means for transferring the reference frames using the plurality of reference frame 
translation tools further comprises: 

a first transfer sub-module for transferring a set of representation/non- 
homogeneous structure coordinates, and 

a second transfer sub-module for transferring a set of non-homogeneous 
structure/representation coordinates. 

4. The interactive system according to claim 3, wherein the first transfer 
sub-module comprises: 

means for acquiring a set of coordinates (XM, YM, ZM), expressed in the 
first reference frame, of a point of the representation of the non-homogeneous structure 
to be transferred, by selection on the representation; 

means for calculating a set of corresponding coordinates (XP, YP, ZP), 
expressed in the second reference frame, on the non-homogeneous structure through a 
transformation: 

{YP,YP, ZP}=M*N.sup.-1 *{XM,YM,ZM} where M * N.sup.-1 represents a 
product of the matrix (M) and an inverse of the matrix (N), and 

means for processing, with the aid of the corresponding coordinates (YP, 
YP, ZP), to display a corresponding point on a surface of the non-homogeneous 
structure and to secure the intervention. 
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5. The interactive system according to claim 3 t wherein the second transfer 
sub-module comprises: 

means for acquiring a set of coordinates (XP, YP, ZP), expressed in the 
second reference frame, of a point of the non-homogeneous structure to be transferred; 

means for calculating a set of corresponding coordinates (XM YM, ZM), 
expressed in the first reference frame, of the representation through a transformation: 

{YM, YM, ZM}=N*M.sup.-1 *{XP,ZP,ZP} where N*M.sup.-1 represents the 
product of the matrix (N) and an inverse of the matrix (M); and, 

means for displaying the representation using the set of corresponding 
coordinates (YM, YM, ZM). 

6. The interactive system according to claim 1, wherein the means for 
generating the plurality of reference frame translation tools also generate, in association 
with the reference frame translation tools, tools for taking into account a residual 
uncertainty which is based on the set of deviations between the set of coordinates of the 
plurality of images of the plurality of base points in the first reference frame and the set 
of coordinates of the base points, the tools for taking into account the residual 
uncertainty usable for displaying a set of contours in the representation whilst taking into 
account the residual uncertainties. 

7. The interactive system according to claim 1, wherein the means of 
dynamic displaying the three-dimensional image comprises: 
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a file containing digitized data from a set of two-dimensional images 
constituted by successive non-invasive tomographic sections of the non-homogeneous 
structure; 

means for calculating and reconstructing the three-dimensional image 
from the set of two-dimensional images; and 
a high-resolution display screen. 

8. The interactive system according to claim 7, wherein the means for 
calculating and reconstructing the three-dimensional image from the set of two- 
dimensional images comprises a program consisting of computer-aided design type 
software. 

9. The interactive system according to claim 1, wherein the means for 
determining the set of coordinates of the plurality of base points in the second reference 
frame comprises a three-dimensional probe equipped with a tactile tip for delivering a 
set of coordinates of the tactile tip in the said second reference frame. 

10. The interactive system according to claim 1, wherein the means for 
determining the set of coordinates of the plurality of base points is the second reference 
frame comprises at least one of a set of optical sensors and a set of electromagnetic 
sensors. 
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1 1 . The interactive system according to claim 1 , wherein a portion of the set of 
the plurality of base points of the reference structure comprises a plurality of marks 
positioned on a lateral surface of the non-homogeneous structure. 

12. The interactive system according to claim 11, wherein the plurality of 
marks are four in number and are distributed over the lateral surface so as to define a 
substantially symmetrical tetrahedron. 

13. The interactive system according to claim 1, wherein the means of 
intervention comprises: 

a guide arm to secure intervention in the region of the non-homogeneous 
structure, the guide arm having a position marked with respect to the second reference 
frame; and, 

an active intervention member whose position is marked with respect to 
the second reference frame. 

14. The interactive system according to claim 13, wherein the active 
intervention member is removable and selected from the group consisting of: 

tools for trephining; 
needles and implants; 

laser and radioisotope emission heads; and, sighting and viewing 

systems. 
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15. The interactive system according to claim 1, wherein the means for 
transferring the plurality of reference frames establishes a coupling between a direction 
of visualization of the representation of the non-homogeneous structure on the display 
means and a direction of observation of the non-homogeneous structure and of the 
reference structure by the active intervention member. 

16. The interactive system according to claim 15, further comprising: 

a first module for visualizing a representation in a direction given by two 

points; 

a second module for visualizing a representation in a direction given by an 
angle of elevation and an angle of azimuth. 

17. (Canceled) 

18. (Canceled) 
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19. (Twice Amended) An interactive system for intervention inside a region 
of a patient, said interactive system comprising: 

a device operable to receive image data of the region of the patient 

wherein the image data includes image data of a first reference structure to establish an 
image reference frame for the region of the patient; 

a second reference structure positioned relative to the patient to establish 

a patient reference frame for the region of the patient; 

a controller operable to correlate the position of the first reference 

structure in the image reference frame with the position of the second reference 
structure in the patient reference frame; 

an active member operable to perform the intervention; and 

a tracking system operable to determine a position of at least the second 

reference structure and a position of the active member and configured to transmit the 
determined positions of the second reference structure and the active member to the 
controller; 

wherein the controller is configured to determine the position of the active 

member based on the determined position of at least the active member and the 
correlation of the first reference structure and the second reference structure. 

20. (previously presented) The interactive system as defined in Claim 19 

wherein the first reference structure includes a plurality of base points. 
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21. (previously presented) The interactive system as defined in Claim 20 

wherein the second reference structure includes a plurality of tracking markers. 

22. (previously presented) The interactive system as defined in Claim 19 

wherein the second reference structure includes a plurality of tracking markers. 

23. (previously presented) The interactive system as defined in Claim 22 

wherein the plurality of tracking markers are attached to the patient. 

24. (previously presented) The interactive system as defined in Claim 19 

wherein the second reference structure is attached to the patient. 

25. (previously presented) The interactive system as defined in Claim 19 

wherein the first reference structure is attached to the patient. 

26. (previously presented) The interactive system as defined in Claim 21 

wherein the plurality of base points are generated from the plurality of tracking markers. 

27, (previously presented) The interactive system as defined in Claim 20 

wherein the plurality of base points are at least one of a plurality of notable points on the 
patient and marks fixed to the patient. 
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28. (previously presented) The interactive system as defined in Claim 27 

wherein the notable points are selected from a group comprising a head, eyebrows, 
temples, frontal medial point, an apex of a skull, a center of gravity of an orbits of the 
eves and a combination thereof. 

29. (previously presented) The interactive system as defined in Claim 19 

wherein the tracking system includes a marker device operable to determine a position 
of the second reference structure in relation to the patient reference frame. 

30. (Amended) The interactive system as defined in Claim 29 wherein the 

marker device includes a telemetry system operable to determine the position of the 
second reference structure in the patient reference frame and transmit the determined 
position to the controller, wherein the controller is operable to perform the correlation at 
least with the transmitted determined position. 

31. (previously presented) The interactive system as defined in Claim 30 

wherein the telemetry system is an electromagnetic telemetry system. 

32. (previously presented) The interactive system as defined in Claim 31 
wherein the second reference structure includes electromagnetic tracking markers, 
wherein the electromagnetic telemetry system is operable to determine the position of 
the electromagnetic tracking markers of the second reference structure in relation to the 
patient reference frame. 
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33. (previously presented) The interactive system as defined in Claim 32, 

wherein the electromagnetic tracking markers are transmitters and the electromagnetic 
telemetry system is an electromagnetic sensor, 

34, (previously presented) The interactive system as defined in Claim 30 

wherein the telemetry system is an optical telemetry system. 

35. (Amended) The interactive system as defined in Claim 34 wherein the 

optical telemetry system includes at least one of a video camera or an infrared camera 
to image at least the second reference structure and configured to plot points of the 
second reference structure. 

36. (previously presented) The interactive system as defined in Claim 34 
wherein the second reference structure includes optical tracking markers, wherein the 
optical telemetry system is operable to determine the position of the optical tracking 
markers of the second reference structure in relation to the patient reference frame. 

37. (previously presented) The interactive system as defined in Claim 34 

wherein the optical telemetry system utilizes position and shape recognition to identify 
the second reference structure. 

38. (previously presented) The interactive system as defined in Claim 29 

wherein the marker device includes a three-dimensional probe. 
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39, (previously presented) The interactive system as defined in Claim 38 
wherein the three-dimensional probe includes a tactile tip operable to engage the 
second reference structure. 

40. (previously presented) The interactive system as defined in Claim 38 

wherein the three-dimensional probe is robotically manipulated, such that the 
instantaneous position of the three-dimensional probe is known. 

41. (previously presented) The interactive system as defined in Claim 29 

wherein the marker device includes a set of cameras operable to determine the position 
of the second reference structure in relation to the patient reference frame. 

42. (previously presented) The interactive system as defined in Claim 41 

wherein the set of cameras are selected from video and infrared cameras. 

43. (previously presented) The interactive system as defined in Claim 29 

wherein the marker device is a laser beam emission system operable to illuminate the 
second reference structure to determine a position of the second reference structure in 
relation to the patient reference frame. 

44. (previously presented) The interactive system as defined in Claim 20 

wherein the controller further includes a graphical tool operable to identify the plurality of 
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base points of the first reference structure in the image data of the image data reference 



frame. 



45. (previously presented) The interactive system as defined in Claim 44 

wherein the graphical tool is a mouse in communication with the controller. 

46. (previously presented) The interactive system as defined in Claim 19 

wherein the first reference structure is generated from the second reference structure. 

47. (canceled) 

48. (Amended) The interactive system as defined in Claim 19 wherein the 

active member is selected from a group comprising a trephining tool, a needle, a laser, 
a radioscope emission head, an endoscopic viewing system, a tool used in the 
intervention, an implant, a sighting system, a microscope, and combinations thereof. 

49. (Amended) The interactive system as defined in Claim 19 further 

comprising a telemetry system operable to determine the position of the active member 
in the patient reference frame, said telemetry system in communication with the 
controller. 
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50, (previously presented) The interactive system as defined in Claim 49 

wherein the position information of the active member is six degree of freedom 
information in relation to the patient reference frame. 

51. (Amended) The interactive system as defined in Claim [[47]] 19 wherein 

the device includes a display operable to display the image data of the region of the 
patient in relation to the image reference frame, 

52, (previously presented) The interactive system as defined in Claim 51 

wherein the controller is further operable to determine a reference origin of intervention 
and a direction of intervention and said display is further operable to display the 
reference origin of intervention and direction of intervention. 

53. (previously presented) The interactive system as defined in Claim 51 

wherein the controller is further operable to model a reference origin of intervention and 
a direction of intervention and said display is further operable to display the modeled 
reference origin of intervention and direction of intervention. 

54. (Amended) The interactive system as defined in Claim 51 wherein the 

display is further operable to display the real-time position of the active member in the 
image reference frame based on the determined position of the active member with the 
tracking system. 
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55. (previously presented) The interactive system as defined in Claim 51 

wherein the display is further operable to display image data relative to a direction of 
intervention of the active member. 

56. (previously presented) The interactive system as defined in Claim 55 

wherein the image data is displayed perpendicular to a direction of intervention of the 
active member. 

57. (previously presented) The interactive system as defined in Claim 51 

wherein the controller is further operable to simulate an optimal trajectory of advance of 
the active member and said display is operable to display the optimal trajectory in the 
image data relative to the image reference frame. 

58. (previously presented) The interactive system as defined in Claim 57 

wherein movement of the active member is steered to the optimal trajectory to carry out 
a programmed intervention. 

59. (Amended) The interactive system as defined in Claim 19 wherein the 

active member is robotically controlled. 

60. (previously presented) The interactive system as defined in Claim 19 

wherein the image data is at least one of a magnetic resonance image data, a 
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tomographic image data, a radiographic image data, x-ray image data, and 
combinations thereof. 

61. (previously presented) The interactive system as defined in Claim 19 

wherein the device is operable to construct three-dimensional images from captured 
two-dimensional images. 
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62. (previously presented) An interactive system for intervention inside a region 
of a patient, said interactive system comprising: 

a device operable to receive image data of the region of the patient, 

wherein the image data includes image data of a first reference structure to establish an 
image reference frame for the region of the patient; 

a second reference structure positioned relative to the patient to establish 

a patient reference frame for the region of the patient; and 

a controller operable to correlate the position of the first reference 

structure in the image reference frame with the position of the second reference 
structure in the patient reference frame; 

wherein the device is operable to construct three-dimensional images from 

captured two-dimensional images; 

wherein the controller is operable to superimpose two-dimensional image 

data on the three-dimensional images wherein any change in soft external parts of the 
patient can be visualized as compared with the image captured by the imaging device. 
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63. (previously presented) An interactive system for intervention inside a 

region of a patient, said interactive system comprising: 

a device operable to receive image data of the region of the patient 

wherein the image data includes image data of a first reference structure to establish an 
image reference frame for the region of the patient; 

a second reference structure positioned relative to the patient to establish 

a patient reference frame for the region of the patient; 

a controller operable to correlate the position of the first reference 

structure in the image reference frame with the position of the second reference 
structure in the patient reference frame; and 

an active member operable to perform the intervention; 

wherein the device includes a display operable to display the image data 

of the region of the patient in relation to the image reference frame; 

wherein the controller is further operable to determine residual uncertainty 

which is used to represent a contour with dimensions larger than those which would 
normally be represented and the display is operable to display the residual uncertainty 
of the contour 

64. (previously presented) The interactive system as defined in Claim 63 

wherein the contour is a display of an active member and a representation of residual 
uncertainty in order to reduce the chance of traversing undesired structures. 
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65, (previously presented) The interactive system as defined in Claim 19 

wherein the controller is further operable to correlate map data in a map reference 
frame with the patient reference frame. 

66. (Amended) The interactive system as defined in Claim 19 wherein the 

intervention is at least one of a neurosurgery, orthopedic surgery, cranial surgery, and 
combinations thereof. 

67. (previously presented) The interactive system as defined in Claim 19 

wherein the second reference structure is fixed to a head set. 

68. (previously presented) The interactive system as defined in Claim 60 

wherein the head set is further fixed to an operating table. 

69. (previously presented) The interactive system as defined in Claim 19 

wherein the device further includes memory operable to store the image data. 

70. (previously presented) The interactive system as defined in Claim 19 

wherein the device is a first computer. 

71. (previously presented) The interactive system as defined in Claim 70 

wherein the controller is a second computer. 
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72. (previously presented) The interactive system as defined in Claim 71 

wherein the first computer and the second computer is a single work station. 
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73. (Twice Amended) An interactive system for intervention inside a region 
of a patient, said interactive system comprising: 

a device operable to receive image data of the region of the patient, 

wherein the image data includes image data of a first reference structure to establish an 
image reference frame for the region of the patient; 

a second reference structure positioned relative to the patient to establish 

a patient reference frame for the region of the patient; 

a controller operable to correlate the position of the first reference 

structure in the image reference frame with the position of the second reference 
structure in the patient reference frame; 

an active member operable to perform the intervention inside the region of 

the patient; 

a tracking system operable to track the position of the active member in 

relation to the patient reference frame, the tracking system being in communication with 
the controller to transmit the tracked position of the active member as position 
information to the controller, wherein the controller is operable to determine the position 
of the active member relative to the image reference frame; and 

a display operable to display the real-time position of the active member in 

the image reference frame based on the controller determined position of the active 
member based on the tracked position of the active member from the tracking system, 
wherein the controller is configured to generate a representation of the active member 
that is displayed on the display relative to a display of the received image data. 
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74. (previously presented) The interactive system as defined in Claim 73 
wherein the active member is selected from a group comprising a trephining tool a 
needle, a laser a radioscope emission head, an endoscopic viewing system, a tool 
used in the intervention, an implant, a sighting system, a microscope, and combinations 
thereof. 

75. (previously presented) The interactive system as defined in Claim 73 

wherein the position information of the active member is six degree of freedom 
information in relation to the patient reference frame. 

76. (previously presented) The interactive system as defined in Claim 

73 wherein the tracking system that tracks the position of the active member is a 
telemetry system in communication with the controller. 

77. (previously presented) The interactive system as defined in Claim 73 

wherein the active member is roboticallv controlled. 

78. (previously presented) The interactive system as defined in Claim 73 

wherein the image data is at least one of a magnetic resonance image data, a 
tomographic image data, a radiographic image data, x-ray image data, and 
combinations thereof. 
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79. (previously presented) The interactive system as defined in Claim 73 

wherein the controller is further operable to determine a reference origin of intervention 
and a direction of intervention and said display is further operable to display the 
reference origin of intervention and direction of intervention. 

80. (previously presented) The interactive system as defined in Claim 73 

wherein the first reference structure includes a plurality of base points. 

81. (previously presented) The interactive system as defined in Claim 80 

wherein the second reference structure includes a plurality of tracking markers. 

82. (previously presented) The interactive system as defined in Claim 81 

wherein the plurality of base points are generated by the plurality of tracking markers. 

83. (previously presented) The interactive system as defined in Claim 73 

wherein the second reference structure is attached to the patient. 

84. (previously presented) The interactive system as defined in Claim 73 

wherein intervention is at least one of a neurosurgery, orthopedic surgery, cranial 
surgery intervention, and combinations thereof. 

85. (previously presented) The interactive system as defined in Claim 73 

wherein the second reference structure is fixed to a head set. 
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86. (Amended) The interactive system as defined in Claim 73 wherein the 

display forms part of the device and wherein the image data received is acquired image 
data of the region of the patient and is displayed on the display, further wherein the 
representation of the active member is displayed on the acquired image data of the 
region of the patient. 
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87. (Twice Amended) A method for performing an image guided 

intervention inside a region of a patient, said method comprising: 

accessing a first image data of the region of the patient captured with an 

imaging system where the first image data includes image data of a first reference 
structure; 

identifying the first reference structure in the first image data to establish 

an image reference frame; 

identifying a second reference structure relative to the patient to establish 

a patient reference frame; 

correlating the position of the first reference structure in the image 

reference frame in the first image data with the position of the second reference 
structure in the patient reference frame; and 

tracking an active member at least to determine a position of the active 

member in the patient reference frame to determine a location of the active member 
based on the tracking of the active member and transmitting the determined position in 
the patient refrence frame for display on a display device relative to the image reference 
frame of the first image data based at least on the correlation of the first reference 
structure and the second reference structure. 

88. (previously presented) The method as defined in Claim 87 further 

comprising attaching a plurality of tracking markers to the patient where the tracking 
markers form the second reference structure. 
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89. (previously presented) The method as defined in Claim 88 further 

comprising identifying the position of the tracking markers in the patient reference frame 
using a telemetry system. 

90. (Amended) The method as defined in Claim 89 further comprising 

transmitting from the tracking markers a signal and receiving the transmitted signal with 
an electromagnetic sensor to identify the position of the second reference structure in 
the patient reference frame. 

91. (previously presented) The method as defined in Claim 87 wherein 

identifying the first reference structure includes identifying a plurality of base points 
visible in the image data. 

92. (previously presented) The method as defined in Claim 91 wherein 

identifying the plurality of base points includes identifying at least one of notable points 
on the patient as marks fixed to the patient representing the plurality of base points. 

93. (previously presented) The method as defined in Claim 92 wherein the 

notable points are selected from a group comprising a head, eyebrows, temporal point, 
frontal medial point an apex of a skull, a center of gravity of an orbits of the eves and a 
combination thereof. 
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94. (previously presented) The method as defined in Claim 91 wherein the 

plurality of base points visible in the image data are generated from the plurality of 
tracking markers attached to the patient. 

95. (previously presented) The method as defined in Claim 87 further 
comprising attaching the second reference structure to the patient. 

96. (previously presented) The method as defined in Claim 87 further 

comprising displaying the image data of the region of the patient, including displaying 
the first reference structure. 

97. (previously presented) The method as defined in Claim 87 further 

comprising performing an intervention on the patient with an active member. 

98. (Canceled) 

99. (Amended) The method as defined in Claim 96 further comprising: 

displaying the position of the active member as a representation of the 

active member in the accessed first image data that is captured image data that is 
correlated to the patient based on the correlation and displayed on a display device with 
the position of the active member being correlated between the patient reference frame 
defined by the first reference structure fixed to the patient and the image reference 
frame based on the tracking of the active member. 
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100. (Amended) The method as defined in Claim 99 further comprising 

identifying the position of the active member with a telemetry system by transmitting the 
tracked location of the active member for displaying the representation of the active 
member. 

101. (previously presented) The method as defined in Claim 99 further 

comprising displaying a reference origin of intervention and a direction of intervention in 
the image data. 

102. (previously presented) The method as defined in Claim 101 further 

comprising tracking the position of the active member relative to the reference origin of 
intervention and the direction of intervention. 

103. (previously presented) The method as defined in Claim 87 further 

comprising converting two-dimensional image data to three-dimensional image data. 

104. (previously presented) The method as defined in Claim 97 wherein the 

intervention is selected from at least one of a neurosurgery, orthopedic surgery, cranial 
surgery, and combinations thereof. 

105. (previously presented) The method as defined in Claim 95 further 

comprising attaching the second reference structure to a head set. 
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ATTACHMENT C 



Taylor, Michael 



From: Kimberly Wallace <kwallace@globlanglinks.com> 

Sent: Tuesday, August 30, 2011 12:37 PM 

To: Taylor, Michael 

Subject: RE: Translation Quote Request (5074A-000013/REA) 




Hi Michael, 

Not a problem. We can get it done for you^^^^^nd deliver by tomorrow. 

Thank you for your business!! 

Best Regards, 

Kimberly Wallace 
Global Language Links LLC 
Office: 248-283-0615 
Fax: 877-800-9526 
kwallace@qloblanQlinks.com 
www.GlobLanaLinks.com 



From: Taylor, Michael [mailto:mltaylor@HDP.com] 
Sent: Tuesday, August 30, 2011 11:48 AM 
To: Kimberly Wallace 

Subject: RE: Translation Quote Request (5074A-000013/REA) 



Thank you, 
Michael 

-Begin text for translation: 
Dear , 

As you may recall, you communicated with Rick Warner or Christopher Eusebi in late 2002 and early 2003 regarding a 
reissue application of U.S. Patent 5,568,675 (attached hereto). At that time you executed a Re-issue declaration. 

During our work with the U.S. Patent office to obtain a grantable patent we amended the claims to those in the attached 
"Current Claims," which we provide for your review. Per the rules of a reissue application, we will file a Supplemental Re- 
issue declaration. We have attached a "Supplemental Reissue Declaration" for your execution. We believe that this 
should be the last paper for this application. Once the U.S. Patent office receives this executed paper a re-issue patent 
should be granted. A re-issue patent essentially replaces the original patent. 

Please execute and return to us the executed "Supplemental Reissue declaration," A return email or fax is acceptable. If 
you can not email or fax an executed copy to us and require returning a physical copy, we can provide a DHL account 
number to cover the expense of shipping a paper copy to us. 

In the table below is the current contact information we have for each of inventor. Please confirm that the current contact 
information is correct. Also, if you can provide us with an email address for Jean-Baptiste Thiebaut we would appreciate it 
very much. 
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We thank you very much in advance for your assistance and quick response. 



:End Text for translation- 
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Michael L. Taylor | Patent Attorney 
O J 248.641,1600 D | 248.641,1289 
IP Causes Worldwide 



F t 248.641.0270 



From: Kimberly Wallace [mailto:kwallace@gIoblanglinks.com] 
Sent: Monday, August 29, 2011 10:29 AM 
To: Taylor, Michael 

Subject: RE: Translation Quote Request (5074A-000013/REA) 
Hi Michael, 

When you say rush does that mean you need then in an hour or two hours or just same day by end of 

business? Turnaround time depends on the availability of my translators. Does that make sense? Typically, we can get it 

done by end of day if not sooner. 

Our min cost is $60 typically for a half page and $120 for a full page and if you need it the same day or "rush" there is an 
additional 20% added on. 

We also offer telephonic translation at $1.85 per minute so if calling would work better I can give you that information; 
the interpreters go from French to English and English to French right on the phone with you via conference call. 

Thank you for the questions and let me know if you have any more and we can work with you. 

Best Regards, 

Kimberly Wallace 

Global Language Links LLC 

Office: 248-283-0615 

Fax: 877-800-9526 

kwallace(S)qloblanqlinks.com 

www.GlobLanqLinks.com 



From: Taylor, Michael [mailto:mltaylor@HDP.com] 
Sent: Sunday, August 28, 2011 11:36 AM 
To: Kimberly Wallace 

Subject: Translation Quote Request (5074A-000013/REA) 



Dear Kimberly, 

I may have need of another English to French Translation. 

What is the general "rush" turn around time and is there an additional cost for a rush order. All translations would be 
about 1/2 to 1 page. 

I may also get communication in French from individuals that I would need rush translated to English. 
Regards, 
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Michael 



Patent Attorney 



Michael L, Taylor 



Office; 
Direct: 
Fax: 



248.641.1600 
248.641.1289 
248.641.0270 




IP Causes Worldwide 



Bio ♦ vCard 



Metropolitan J Detroit ♦ Portland * St. Louis ♦ Washington, DC 



HARNESS, DICKEY & PIERCE, PLC LEGAL NOTICE: The information contained in this transmission is intended only for the individuat(s) or entily(ies) to 
whom it is addressed. It may contain information protected from use and/or disclosure by law, Including Information that is protected as confidential, attorney-client 
privileged, attorney work product and/or trade secrets. If the reader of this message is not the intended recipient, or an employee or agent responsible for 
delivering this message to the addressee, the reader is hereby notified that any use, distribution or copying of this communication is strictly prohibited, if you 
believe you have received this facsimile or message in error, please immediately notify us at our expense by return mail or e-mail and permanently delete or 
destroy afi copies of the message. 

To comply with U.S. Treasury regulations, we advise you that any discussion of Federal tax issues in this communication was not intended to be used, and 
cannot be used, by any person (i) to avoid penalties that may be imposed by the Internal Revenue Service, or (ii) to promote, market or recommend to another 
party any matter addressed herein. 

Any portions of this transmission containing controlled technical data are restricted by U.S. export laws and regulations, and may not be distributed or 
retransmitted to non-U. S. persons without appropriate licensing or a licensing exemption. Neither this information block, nor the signature block, nor the typed 
name of the sender, nor anything else in this message is intended to constitute an electronic signature unless a specific statement to the contrary is included. 
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ATTACHMENT D 



| 



Taylor, Michael 



From: Taylor, Michael <mltaylor@HDP.com> 

Sent: Tuesday, September 06, 2011 9:42 AM 

To: Michel Scriban 

Cc: Warner, Rick 

Subject: RE: Re-issue of U.S. Patent 5,868,675 (5074a-000013/REA) 



Dear Michel, 

Thank you for confirming receipt of my previous email. 

If everyone is able to return to us the executed declaration by the end of September, that is acceptable timing. 

Did you believe that Dr Thiebaut or JF Uhl would be able to contact Dr. Henrion regarding the declaration? 

The declaration is a formality of U.S. Patent Practice. It indicates that any corrected errors were not 
intentional. The previous declaration was to the same effect. Though you are not the direct authors (i.e. 
we drafted the claims) they are based on the original patent application of which all of you are the inventors, 
therefore you are the authors by extension. 

The current assignee of the 5,868,675 patent is MEDTRONIC, INC. at 7000 CENTRAL AVENUE, N.E., 
MINNEAPOLIS, MINNESOTA 55432 assigned from Elekta IGS S.A. 

I am not certain what you mean by "returns of this validation job." Once the Declarations are executed and 
filed, the patent should re-issue with the greater number of claims than the original patent and which we 
previously forwarded to you. The re-issued patent expire on the same date as the previous 5,868,675 patent. 

Thank you, 

Michael 



M'A'RMPcrc; t^^S Michael L. Taylor | Patent Attorney 



From: Michel Scriban [mailto:m.scriban@nelixa.fr] 
Sent: Monday, September 05, 2011 2:34 AM 
To: Taylor, Michael 

Subject: Re: Re-issue of U.S. Patent 5,568,675 (5074a-000013/REA) 
Hello Michael, 

Hope to be in Paris Mid-September and meet Dr Thiebaut coming back from vacation and perhaps JF Uhl... 

Today, my question is why modifications need our signature as we are not the authors of the new claims. What 
returns can we expect of this validation job ? Who is now the "owner" of this patent ? 

Thanks for your questions. I will transmit your answer to my collegues. 
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Best regards, 

Le 3 1 aout 201 1 a 21 :06 5 Taylor, Michael a ecrit : 



Dr. Scrtban, 

As we discussed yesterday, I am forwarding this email that discusses the U.S. Patent Application that is a reissue of the 
above referenced U.S. Patent and papers that we need executed. I am also copying each inventor for which we 
previously had an email address, Joel Henrion and Joel Francois UhL You indicated that you would forward this email 
to Jean-Baptiste Thiebaut, and also to Joel Henrion and Joel Francois Uhl if the email addresses we have for them are no 
longer correct. We previously had translated the below message that discusses the current process and our reason for 
contacting you. Also, if you have an email address for Jean-Baptiste Thiebaut, we would very much appreciate receiving 
it from you. 

We very much appreciate your assistance in this matter. Please let us know any one of you will not be able to 
return the executed paper to us by October 3 5 201 1. 



Comme vous pouvez vous en souvenir, vous avez ete en contact avec Rick Warner ou Christopher Eusebi fin 2002 et 
debut 2003 concernant une demande de redelivrance du brevet U.S. 5 568 675 (ci-joint). A ce moment-la, vous aviez 
execute une declaration de redelivrance. 



Lors de votre travail avec le bureau des brevets U.S. pour obtenir un brevet subventionnable, nous avons amende les 
reclamations par celles jointes dans les « reclamations actuelles » ci-jointes, que nous vous soumettons pour 
revision. Selon les regies regissant les demandes de redelivrance, nous allons deposer une declaration de redelivrance 
additionnelle. Nous avons joint une « declaration de redelivrance additionnelle » pour votre execution. Nous pensons qu'il 
s'agit du dernier document necessaire pour cette demande. Une fois que le bureau des brevets U.S. regoit ce document 
dument execute, une redelivrance de brevet devrait etre accordee. Un brevet redelivre remplace essentiellement le brevet 
d'origine. 



Veuillez dument executer la « declaration de redelivrance additionnelle » et nous la renvoyer. Vous pouvez nous 
I'envoyer par courriel ou par fax. Si vous ne pouvez nous envoyer pas courriel ou par fax une copie dument ex^cutee et 
devez envoyer une copie physique, nous pouvons vous donner un numero de compte DHL pour couvrir le cout de 
1'envoi. 



Dans le tableau ci-dessous, vous trouverez les coordonnees que nous avons pour chacun des inventeurs. Veuillez 
confirmer que ces informations sont exactes. Si vous pouviez egalement nous donner I'adresse courriel de Jean-Baptiste 
Thiebaut, nous en serions tres reconnaissants. 



Nous vous remercions d'avance pour votre aide et votre reponse rapide. 
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Name 


DHL/Mailing Address 


Email Address(es) 


Phone 


Michel Scriban 


72 Chemin de Crapon 
69360 Ternay, France 


m.scriban@nelixa,fr 




Joel Henrion 


17, Route de Chalone 
51600 Suippes, France 


Joel.henrion(3iwanadoo.fr 




Jean Francois UHL 


199 avenue du Maine 
Paris, France 75014 

(auxiliary address- 12 rue Regard; 92380 Garche, 
France) 


Jf.uhl@free.fr 
Jf.uhl@wanadoo.fr 




Jean-Baptiste Thiebaut 


42 boulevard Saint-Marcel 
Paris, France 75005 







Thank you and Best Regards, 



Michael 



[ Michael L. Taylor 

| Patent Attorney 
I <at±7e9cd.qif> 



IP Causes Worldwide 



Office: 248.641.1600 
Direct: 248.641.1289 
Fax: 248.641.0270 

5445 Corporate Dr, Suite 
200 

Troy, MI 48098 

Bto ♦ vCard 



Metropolitan j Detroit ♦ Portland ♦ St. Louis ♦ Washington, DC 



HARNESS, DSCKEY & PIERCE, PLC LEGAL NOTICE: The information contained in this transmission is intended only for the individuai(s) or entity(ies) to 
whom it is addressed. It may contain information protected from use and/or disclosure by law, including information that is protected as confidential, attorney-client 
privileged, attorney work product and/or trade secrets. If the reader of this message is not the intended recipient, or an employee or agent responsible for 
delivering this message to the addressee, the reader is hereby notified that any use, distribution or copying of this communication is strictly prohibited, if you 
believe you have received this facsimile or message in error, please immediately notify us at our expense by return mall or e-mai) and permanently delete or 
destroy ail copies of the message. 

To comply with U.S. Treasury regulations, we advise you that any discussion of Federal tax issues in this communication was not intended to be used, and 
cannot be used, by any person (i) to avoid penalties that may be imposed by the internal Revenue Service, or (it) to promote, market or recommend to another 
party any matter addressed herein. 

Any portions of this transmission containing controlled technical data are restricted by U.S. export laws and regulations, and may not be distributed or 
retransmitted to non-U. S. persons without appropriate licensing or a licensing exemption. Neither this information block, nor the signature block, nor the typed 
name of the sender nor anything else in this message is intended to constitute an electronic signature unless a specific statement to the contrary is included. 

Supplemental Reissue declaration.PDF><US 5 868675. PDF><Current Claims.DOO 
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ATTACHMENT E 



Taylor, Michael 



From: 
Sent: 
To: 
Cc: 

Subject: 



Taylor, Michael 

Wednesday, October 05, 2011 9:16 AM 
'Michel ScribarV 

Jean-BaptisteTHIEBAUT Jean-BaptisteTHIEBAUT 

RE: State of situation - Re: Re-issue of U.S. Patent 5,868,675 (5074a-000013/REA) 



Importance: 



High 



Dear Dr. Scriban, 

Following is a French translation of a letter that we prepared in response to your email below. We appreciate your 
assistance in this matter. 

Nous apprecions que vous ayez pris le temps et fait les demarches necessaires pour contacter Dr. Uhl et 
Dr. Thiebaut. Nous esperons qu'ils vont bien. Nous supposons que tous les trois vous avez pu signer la 
declaration pour redelivrance que nous vous avions transmise. Si c'est le cas, veuillez s'il-vous-plait me 
les envoyer, si possible par courriel. 

Nous apprecions egalement vos efforts pour contacter Mr. Henrion. Veuillez nous prevenir dans le cas oti 
vous ne seriez pas entre en contact avec Mr. Henrion avant le 18 octobre. Avez-vous une idee de la 
fagon dont nous pourrions joindre Mr. Henrion ? La derniere adresse que vous avez pour lui est-elle : 
17, Route de Chalone 
51600 Suippes, France 

etson adresse courriel : Joel.henrion@wanadoo.fr? 

Notre societe est un cabinet d'avocats qui represente Medtronic et Medtronic travaille actuellement a la 
redelivrance du brevet, invente par vous et vos collegues, qui desormais leur appartient. Medtronic est 
maintenant le cessionnaire enregistre et le proprietaire de ce brevet. Vos signatures sur la declaration de 
redelivrance sont pour indiquer que les erreurs corrigees par cette declaration de redelivrance de brevet 
n'ont pas ete commises avec ('intention de tromper de votre part. L'«erreur» a corriger est que les 
revendications ne sont pas aussi larges que ce qui avait ete pense au depart et que la revendication 14 
se limitait a la visualisation endoscopique. Pour Pinstant, Medtronic ne demande rien de plus que votre 
signature sur la declaration de redelivrance pour faciliter ('acceptation de cette declaration qui est bas6e 
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sur le brevet 5 868 675 sur lequel votre nom et ceux de vos collegues apparaissent. 

Nous vous demandons simplement de confirmer que cette phrase dans la declaration «Toute erreur dans 

le brevet qui a ete corrigee dans cette demande de redelivrance et qui n'est pas couverte par le present 

serment ou declaration soumis a cette demande, s'est produite sans intention de tromper de la part de 

Pappliquant » est correcte et de signer cette declaration. Cela signifie essentiellement que toutes les 

erreurs identifies et corrigees par cette declaration ont ete faites sans intention de tromper. En d'autres 

termes, toutes les erreurs trouvees n'etaient pas destinees a tromper quelqu'un. 

Si vous avez des questions, n'hesitez pas a nous en informer. 

Best Regards, 
Michael 



MARMPCc; »Bi Michael t. Taylor [ Patent Attorney 

" O 1 248.641.1600 D j 248.641.1289 F 1 248.641.0270 

□ ICKEY IP Causes Worldwide 



From: Michel Scriban [mailto:m.scriban@nelixa.com] 
Sent: Monday, October 03, 2011 6:12 AM 
To: Taylor, Michael 

Cc: Jean-BaptisteTHIEBAUT Jean-BaptisteTHIEBAUT 

Subject: State of situation - Re: Re-issue of U.S. Patent 5,868,675 (5074a-000013/REA) 
Bonjour, 

Apres un mois de septembre a Femploi du temps extremement charge pour tous et notamment le Dr Thiebaut, 
neurochirurgien, j'ai pu faire ce W.E. un point avec lui quant a votre demande apparu 8 ans apres notre 
precedent contact. 

Pas de pb pour avoir le Dr UHL mais cela n'est pas simple pour Mr Henrion avec qui on n'est pas en contact 
regulier depuis tres longtemps (on cherche) et qui compte tenu de sa formation mathematique etait au coeur des 
cles et algorithmes du brevet initial dont j'etais a Tepoque en charge de la coordination. 

La principale question qui reste en suspens pour tous pour ce brevet dont on a perdu le 'controle' et de 
comprendre quels sont les avantages (intellectuels, financiers,...) de notre signature voire les inconvenients et 
risques. Qu'attend Medtronic des auteurs ? Quel est le role de votre organisation ? 

De plus a ce jour, nous n'avons aucun moyen de faire viser ces textes par un expert en brevet pour valider les 
pretentions que vous nous soumettez... 

Voila notre etat d'esprit a ce jour, 
A tres bientot 
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Michel SCRIBAN m.d. 
E-Mail: m.scriban@nelixa.com 
http://www.nelixa.com/ 



Mail Address 

NelixA 

Espace DMCI - 8ieme Et. 
4 quai des Etroits, 
F 69005 Lyon 
France 

Sec. Tel. +33 4 72 56 51 36 
E-mail: contact@nelixa.com 



Le 6 sept 201 1 a 15:41, Taylor, Michael a ecrit : 



Dear Michel, 

Thank you for confirming receipt of my previous email. 

If everyone is able to return to us the executed declaration by the end of September, that is acceptable timing. 
Did you believe that Dr Thiebaut or JF Uhl would be able to contact Dr. Henrion regarding the declaration? 
The declaration is a formality of U.S. Patent Practice. It indicates that any corrected errors were not 
intentional. The previous declaration was to the same effect. Though you are not the direct authors (i.e. 
we drafted the claims) they are based on the original patent application of which all of you are the inventors, 
therefore you are the authors by extension. 

The current assignee of the 5,868,675 patent is MEDTRONIC, INC. at 7000 CENTRAL AVENUE, N.E., 
MINNEAPOLIS, MINNESOTA 55432 assigned from Elekta IGS S.A. 

I am not certain what you mean by "returns of this validation job." Once the Declarations are executed and 
filed, the patent should re-issue with the greater number of claims than the original patent and which we 
previously forwarded to you. The re-issued patent expire on the same date as the previous 5,868,675 patent. 
Thank you, 
Michael 

Michael L. Taylor j Patent Attorney 
<att65c3e.gjf> O | 248.641. 1600 D | 248.641.1289 F ! 248.641.0270 

IP Causes Worldwide 



From: Michel Scriban [mailto:m.scriban@nelixa.fr] 
Sent: Monday, September 05, 2011 2:34 AM 
To: Taylor, Michael 

Subject: Re: Re-issue of U.S. Patent 5,568,675 (5074a-000013/REA) 
Hello Michael, 

Hope to be in Paris Mid-September and meet Dr Thiebaut coming back from vacation and perhaps JF Uhl.. 

Today, my question is why modifications need our signature as we are not the authors of the new claims. What 
returns can we expect of this validation job ? Who is now the "owner" of this patent ? 
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Thanks for your questions. I will transmit your answer to my collegues. 
Best regards, 

Le 3 1 aout 201 1 a 21 ;06, Taylor, Michael a ecrit : 



Dr. Scriban, 

As we discussed yesterday, I am forwarding this email that discusses the U.S. Patent Application that is a reissue of the 
above referenced U.S. Patent and papers that we need executed. I am also copying each inventor for which we 
previously had an email address, Joel Henrion and Joel Francois Uhl. You indicated that you would forward this email 
to Jean-Baptiste Thiebaut, and also to Joel Henrion and Joel Francois Uhl if the email addresses we have for them are no 
longer correct. We previously had translated the below message that discusses the current process and our reason for 
contacting you. Also, if you have an email address for Jean-Baptiste Thiebaut, we would very much appreciate receiving 
it from you. 

We very much appreciate your assistance in this matter. Please let us know any one of you will not be able to 
return the executed paper to us by October 3 5 201 1. 



Comme vous pouvez vous en souvenir, vous avez ete en contact avec Rick Warner ou Christopher Eusebi fin 2002 et 
debut 2003 concernant une demande de redelivrance du brevet U.S. 5 568 675 (ci-joint). A ce moment-la, vous aviez 
execute une declaration de redelivrance. 

Lors de votre travail avec le bureau des brevets U.S. pour obtenir un brevet subventionnable, nous avons amende les 
reclamations par celles jointes dans les « reclamations actuelles » ci-jointes, que nous vous soumettons pour 
revision. Selon les regies regissant les demandes de redelivrance, nous allons deposer une declaration de redelivrance 
additionnelle. Nous avons joint une « declaration de redelivrance additionnelle » pour votre execution. Nous pensons qu'il 
s'agit du dernier document necessaire pour cette demande. Une fois que le bureau des brevets U.S. re?oit ce document 
dument execute, une redelivrance de brevet devrait etre accordee. Un brevet redelivre remplace essentiellement le brevet 
d'origine. 

Veuillez dument executer la « declaration de redelivrance additionnelle » et nous la renvoyer. Vous pouvez nous 
I'envoyer par courriel ou par fax. Si vous ne pouvez nous envoyer pas courriel ou par fax une copie dument executee et 
devez envoyer une copie physique, nous pouvons vous donner un numero de compte DHL pour couvrir le cout de 
I'envoi. 

Dans le tableau ci-dessous, vous trouverez les coordonnees que nous avons pour chacun des inventeurs. Veuillez 
confirmer que ces informations sont exactes. Si vous pouviez egalement nous donner Padresse courriel de Jean-Baptiste 
Thiebaut, nous en serions tres reconnaissants. 



Nous vous remercions d'avance pour votre aide et votre reponse rapide. 



Name 


DHL/Mailing Address 


Email Address(es) 


Phone 


Michel Scriban 


72 Chemin de Crapon 
69360 Ternay, France 


m.scriban(5)xielixa.fr 




Joel Henrion 


17, Route de Chalone 
51600 Suippes, France 


Joel.henrionOwanadoo.fr 




Jean Francois UHL 


199 avenue du Maine 
Paris, France 75014 

(auxiliary address- 12 rue Regard; 92380 Garche, 
France) 


Jf.uhlOfree.fr 
Jf.uhlOwanadoo.fr 
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Jean-Baptiste Thiebaut 


42 boulevard Saint-Marcel 








Paris, France 75005 







Thank you and Best Regards, 
Michael 



j Michael L. Taylor 

! Patent Attorney 



Office: 248.641.1600 
Direct: 248.641.1289 
Fax: 248.641.0270 



<att7e9cd.oif> 



IP Causes Worldwide 



5445 Corporate Dr, Suite 
200 

Troy; MI 48098 

Bio ♦ vCard 



Metropolitan | Detroit ♦ Portland ♦ St. Louis ♦ Washington, DC 



HARNESS, D8C81EY & PSEKCE, PLC LEGAL NOTICE: The information contained in this transmission is intended only for the individual(s) or entity(ies) to 
whom it is addressed, It may contain information protected from use and/or disclosure by law, including information that is protected as confidential, attorney-client 
privileged, attorney work product and/or trade secrets, if the reader of this message is not the intended recipient, or an employee or agent responsible for 
delivering this message to the addressee, the reader is hereby notified that any use, distribution or copying of this communication is strictly prohibited. If you 
believe you have received this facsimile or message in error, please immediately notify us at our expense by return mail or e-mail and permanently delete or 
destroy ail copies of the message. 

To comply with U.S. Treasury regulations, we advise you that any discussion of Federal tax issues in this communication was not intended to be used, and 
cannot be used, by any person (i) to avoid penalties that may be imposed by the Internal Revenue Service, or (ii) to promote, market or recommend to another 
party any matter addressed herein. 

Any portions of this transmission containing controlled technical data are restricted by U.S. export laws and regulations, and may not be distributed or 
retransmitted to non-U. S, persons without appropriate licensing or a licensing exemption. Neither this information block, nor the signature block, nor the typed 
name of the sender, nor anything else in this message is intended to constitute an electronic signature unless a specific statement to the contrary is included. 



Supplemental Reissue declaration.PDF><US 5 868675. PDFxCurrent Claims.DOO 
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ATTACHMENT F 



Taylor, Michael 



From: Taylor, Michael <mltaylor@HDPxom> 

Sent: Tuesday, October 04, 2011 11:33 AM 

To: globallanguagelinks@yahoo.com; kwallace@globlanglinks.com 

Subject: English to French translation (5074A-000013/REA) 



Kimberly, 

We request that you translate the following into French by tomorrow morning' 
confirm whether this is acceptable or not. 




rush. Please 



Language for Translation begins 

Dear Dr. Scriban, 

We appreciate your effort in contacting Dr. Uhl and Dr. Thiebaut We hope that they are doing well. We assume that all 
three of you have been able to execute the re-issue declaration that we previously forwarded to you. If so, please forward 
them to me, via email if possible. 

We also appreciate your efforts in attempting to contact Mr. Henrion. Please let us now if you are unable to contact Mr. 
Henrion by October 18. Do you have any ideas where we could contact Mr. Henrion? Is the last best address you have for 
Mr. Henrion: 

17, Route de Chalone 

51600 Suippes, France 

And email: Joel.henrion@wanadoo.fr 

Our company is a law firm that represents Medtronic and Medtronic is current woring on the re»issue of the patent that 
they now own, which was originally invented by you and your co-inventors. Medtronic is now the recorded assignee and 
owner of this patent. Your signatures on the reissue declaration are to indicate that any errors corrected by this re-issue 
patent application were not done with any deceptive intent on your parts. The "error" to be corrected is that the claims are 
not as broad as could have originally been draft and that Claim 14 was limited to endoscopic viewing. As of right now, 
Medtronic asks no more than your signature on the re-issue declaration to expedite grant of this application which is 
based on the patent 5,868,675 on which you and your co-inventors are listed. 

We simply ask that you confirm that statement in the declaration "Toute erreur dans le brevet qui a ete corrigee dans cette 
demande de redelivrance et qui n'est pas couverte par le present serment ou declaration soumis a cette demande, s'est 
produite sans intention de tromper de la part de Pappliquant" is correct and execute the declaration. Essentially, any errors 
identified and corrected by these claims were made without deceptive intent. In other words, any identified errors were not 
intended to deceive anyone. 

If you need any further clarification, please let us know. 

Best Regards, 

Michael 



language for translation ends 



l 



Thanks, 
Michael 



Michael L. Taylor j Patent Attorney 
0(248.641.1600 D|248.641.1289 F| 248.641.0270 
5445 Corporate Dr, Suite 200, Troy, MI 48098 

HARNESS. w^zJjM 

IP Causes Worldwide Bio ♦ vCard 

Metropolitan | Detroit * Portland ♦ St. Louis ♦ Washington, DC 



HARNESS, DICKEY & PIERCE, PLC LEGAL NOTICE: The information contained in this transmission is intended only for the individual(s) or entity(ies) to 
whom it is addressed. It may contain information protected from use and/or disclosure by law, including information that is protected as confidential, attorney-client 
privileged, attorney work product and/or trade secrets. If the reader of this message is not the intended recipient, or an employee or agent responsible for 
delivering this message to the addressee, the reader is hereby notified that any use, distribution or copying of this communication is strictly prohibited. If you 
believe you have received this facsimile or message in error, please immediately notify us at our expense by return mail ore-mail and permanently delete or 
destroy all copies of the message. 

To comply with U.S. Treasury regulations, we advise you that any discussion of Federal tax issues in this communication was not intended to be used, and 
cannot be used, by any person (i) to avoid penalties that may be imposed by the internal Revenue Service, or (ii) to promote, market or recommend to another 
party any matter addressed herein. 

Any portions of this transmission containing controlled technical data are restricted by U.S. export laws and regulations, and may not be distributed or 
retransmitted to non-U. S. persons without appropriate licensing or a licensing exemption. Neither this information block, nor the signature block, nor the typed 
name of the sender, nor anything else in this message is intended to constitute an electronic signature unless a specific statement to the contrary is included. 
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ATTACHMENT G 



Taylor, Michael 



From: 

Sent: 

To: 

Subject: 



Michel Scriban <m.scriban@nelixa.com> 
Sunday, October 09, 2011 5:19 PM 
Taylor, Michael 

in progress - Re: Re-issue of U.S. Patent 5,868,675 (5074a-000013/REA) 



Importance: 



High 



Dear Michael, 

As previous, this email will be continued in french because contains important terms. 

Meme si ce n'est pas aise compte-tenu de Femploi du temps tres charge des autres co-auteurs du brevet, nous 
avons pu nous concerter et nous entretenir ensemble quant a votre attente pressente de nous voir signer la 
version amendee du brevet. 

Comme indique dans les precedents emails, une signature n'est pas un geste anodin qui engagent les signataires 
et auteurs. De plus, verifier que Tessentiel est confonne aux revendications initiates et que - selon votre 
expression - vous n'avez pas introduit de "tromperie" va demander que nous etudions - avec Taide d'experts - les 
documents soumis au regard des elements mis au point il y a plus de 10 ans. 

En nous sollicitant, cela rappelle aussi rimportance de ce brevet et les enjeux que nous avions anticipes a cette 
epoque et observes depuis chez Medtronic ou autres industriels. 

Sur le plan pratique, en premiere estimation, la valeur de intervention sera de 17 700 EUR a verser a chacun 
des 4 auteurs soit un total de 70 800 EUR hors frais de transfer! et de sequestre aupres d f un avocat siegant en 
France et retenu par les deux parties. 

Avant de preciser les modalites et affectations, nous vous prions de consulter le proprietaire de ce brevet dont 
nous quatre sommes les auteurs et de nous tenir informe de la suite. 

En vous souhaitant bonne reception, 

Best regards 



For the team of the co-authors 

Michel SCRIBAN m.d. 
E-Mail: m.scriban@nelixa.com 



Le 5 oct. 201 1 a 15:15, Taylor, Michael a ecrit : 



Dear Dr. Scriban, 



l 



Following is a French translation of a letter that we prepared in response to your email below. We appreciate your 
assistance in this matter. 
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ATTACHMENT H 



I 



Dear Michael, 

As previous, this email will be continued in French because contains important terms. 

Although this was not easy because of the busy schedule of the other co-authors of the patent, we were 
able to meet and to talk about your expectations to see us sign the amended Patent 

As mentioned in previous emails, a signature is not insignificant and it commits the signatories and 
authors. Also, verifying that the summary is consistent with the original claims and that - as you say - 
you did not introduce a "deception" will require that we study - with the help of experts - the 
documents submitted with respect to the elements developed more than 10 years ago. 

The fact that you are asking our signatures also emphasizes the importance of this patent and the issues 
that we had anticipated at that time and that have since been discovered at Medtronic and at other 
manufacturers. 

In practical terms, as an initial estimate, the value of the intervention will be 17,700 EUR to be paid to 
each of the four authors for a total of 70,800 EUR excluding the fees for transfer and for sequestration 
by a lawyer based in France and retained by both parties. 

Before specifying the terms and assignments, please consult the owner of this patent for which the four 
of us are the authors and keep us informed of the outcome. 



Wishing you good reception, 



ATTACHMENT 



I 



HARNESS 

DICKEY 



Michael L. Taylor 

Direct Dial: 248-641-1289 
mitaylor@hdp.com 



October 31, 2011 





Michel Scriban 
72 Chemin de Crapon 
69360 Ternay, France 
m.scriban@nelixa.fr 

Joel Henrion 
17, Route de Chalone 
51600 Suippes, France 
Joel, henrion@wanadoo.fr 



Jean Francois UHL 

199 avenue du Maine 

Paris, France 75014 

Jf.uhl(g)free.fr and Jf.uhl@wanadoo.fr 

Jean-Baptiste Thiebaut 
42 boulevard Saint-Marcel 
Paris, France 75005 
jbthiebaut@fo-rothschild.fr 



VIA DHL and EMAIL 



Waybill: 8809998164 — Date: 2011-11-01 — Pieoes: 1/1 — 
Description: BUSINESS DOCUMENTS — Postcode: 75005, FRANCE ~ 
Service: DOX — Billing. To: Sender — Customs: NVD — Shipment 
Weight: 0.7 lb — Discounted Total Charge: $30,90 



Waybill: 8809998153 — Date: 2011-11-01 — Pieces: 1/1 — 
Description: BUSINESS DOCUMENTS — Postcode: 75014, FRANCE — 
Service: DOX — Billing To: Sender — Customs: NVD — Shipment 
Weight: 0.7 lb Discounted Total Charge: $30.90 



Waybill: 8809998142 — Date: 2011-11-01 — Pieces: 1/1 — 
Description: BUSINESS DOCUMENTS — Postcode: 51600, FRANCE — 
Service: DOX — Billing To: Sender — Customs: NVD — Shipment 
Weight: 0.7 lb — Discounted Total Charge: $30.90 



Waybill: 8809998131 — Date: 2011-11-01 — Pieoes: 1/1 — 
Description: BUSINESS DOCUMENTS — Postcode: 69360, FRANCE — 
Servioe: DOX -- Billing To: Sender — Customs: NVD — Shipment 
Weight: 0.7 lb — Discounted Total Charge: $30.90 



Ref : 
Dep: 



OFFICE 
Michael 



Date: 
Taylo Wgt ; 

DV: 



01Nov11 
1.00 LBS 



0.00 



SHIPPING: 
SPECIAL: 
HANDLING: 
TOTAL: 



Svcs i^J ftN i>ARD OVERNIGHT 
TRCK : 9061 5385 6863 



Re: U.S. reissue application^ 
Reissue of the U oepi 
Medtronic Ref. No. PCO 
HDP Ref. No. 5074A-0C 



OFFICE 
Michael L, 



Date: 01Nov11 
Taylo Wgt : 1 .00 LBS 

DV: 



0.00 



SHIPPING; 
SPECIAL: 
HANDLING: 
TOTAL: 



Svc ^£ TA NPA R P OVERNIGHT 
TRCK; 9061 5385 6872 



9.92 
1.44 
0.00 
11.36 



9.92 
1.44 
0.00 
11.36 



Dear Inventors, 

This communication is being provide 
translation. 



Ref: OFFICE Date: 01Nov11 

Dep: Michael L. Taylo Wgt : 1.00 LBS 

DV: 



SHIPPING: 
SPECIAL: 
HANDLING: 
0.00 TOTAL: 



Svcs; STANDARD OVERNIGHT 
TRCK : 9061 5385 6861 



Ref: OFFICE Date: 01Nov11 

Dep: Michael L. Taylo Wgt: 1.00 LBS 

DV: 



SHIPPING: 
SPECIAL: 
HANDLING: 
0.00 TOTAL: 



Svc ^ S J AN i^ OVERNIGHT 
TRCK: 9061 5385 6894 



9.92 
1 .44 
0.00 
11 .36 



9,92 
1.44 
0.00 
11.36 



Harness, Dickey & Pierce PLC 
Attorneys and Counselors 
P.O. Box 828 

Bioomfield Hills, Michigan 48303 U.S.A 
248.641.1600 - fax 248.641.0270 



FOR COURIER DELIVERY ONLY 
5445 Corporate Drive, Suite 200 
Troy, Michigan 48098 U.S.A. 



Metropolitan: 

Detroit, Ml St. Louis, MO Washington, D.C. Portland, OR 




Michael L Taylor 

Direct Dial: 248-641-1289 
mitayior@hdp.com 



November 1, 2011 



VIA DHL and EMAIL 



Michel Scriban 

72 Chemin de Crapon 

69360 Ternay, France 

m.scriban@nelixa.fr 

Joel Henrion 

17, Route cle Chalone 

51600 Suippes, France 

JoeLhenrion@wanadoo.fr 

Jean Francois UHL 

199 avenue du Maine 

Paris, France 75014 

Jf.uhl@free.fr and Jf.uhl@wanadoo.fr 

Jean-Baptiste Thiebaut 
42 boulevard Saint-Marcel 
Paris, France 75005 
jbthiebaut@fo-rothschild.fr 



Re: U.S. reissue application 09/784,829 which is a 
Reissue of the U.S. Patent 5,868,675 
Medtronic Ref. No. PC00001 73.06 
HDP Ref. No. 5074A-0.00013/REA 



Dear Inventors, 



This communication is being provided in English followed by a French 



translation. 



Harness, Dickey & Pierce PLC 
Attorneys and Counselors 

P.O. Box 828 FOR COURIER DELIVERY ONLY 

Bioomfieid Hills, Michigan 48303 U.S.A 5445 Corporate Drive, Suite 200 

248.6411600 - fax 248.641.0270 Troy, Michigan 48098 U.S.A. 



Metropolitan: 

Detroit, MI St. Louis, MO Washington, D.C. Portland, OR 



November 1, 2011 
Page 2 of 8 

Medtronic, Inc. (herein Medtronic) hereby respectfully requests that each 
individual named as an inventor in the U.S. reissue application 09/784,829, including 
Michel Scriban, Joel Henrion, Jean Francois UHL, and Jean-Baptiste Thiebaut, execute 
the previously provided Supplement Reissue Declaration and return it to Medtronic's 
representative Richard W. Warner at Harness, Dickey & Pierce promptly via email at 
rwarner@hdp.com or mail/courier to Harness Dickey, 5445 Corporate Dr, Suite 200, 
Troy, Ml 48098 by November 14, 2011. For your convenience, we have provided pre- 
paid packages to return to us the executed Supplemental Reissue Declaration. 

Medtronic has reviewed your letter of October 1 0, 20 11 that requests payment of 
70,800 EUR (about US$ 98,743.68) to all of the inventors for intervention, including 
execution of the previously provided Supplemental Reissue Declaration. Medtronic 
notes that no payment is required under the assignment agreement that was executed 
by all of the inventors on June 22, 1992 and deems your request for payment a refusal 
to execute and return the Supplemental Reissue Declaration. Medtronic would like to 
take this opportunity to fully explain why no payment is necessary and provide each of 
the inventors one final opportunity to execute and return to us the Supplemental 
Reissue Declaration. 

An Assignment (Under Tab A) to Diadix S.A. that was executed by each of the 
inventors on June 22, 1992 states, "The undersigned hereby agree(s) to transfer a like 
interest, upon request of the said ASSIGNEE, its successors, assigns and legal 
representatives, and without further remuneration, in and to any and all divisions, 
continuations, substitutes, and reissues thereof; and to testify and execute any 
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papers for ASSIGNEE, its successors, assigns and legal representations, deemed 
essential by ASSIGNEE to ASSIGNEE'S full protection and title in and to the invention 
hereby transferred." Thus, Medtronic believes that executing further documents for 
reissue applications is a duty of each inventor upon Medtronic's request and should 
occur without further remuneration (i.e. payment). 

The original U.S. Patent 5,868,675 was assigned from Diadix S.A. to Deemed 
International (executed on October 24, 1995), then from Deemed International to Elekta 
IGS S.A. (via a name change that was executed on August 14, 1997), and finally from 
Elekta IGS S.A. to Medtronic, Inc. (executed on December 14, 1999). Each of these 
Assignments is duly recorded in the United States Patent and Trademark Office in the 
U.S. Patent No. 5,868,675. The chain of title, as shown by the United States Patent 
and Trademark Office, is included under Tab B and can be viewed at 
http://assignments.uspto.aov/assignments/q?db=pat&qt=pat&reel=&frame=&pat=5868 
675 . The subject application, to which the Supplement Reissue Declaration relates, is 
a reissue of the U.S. Patent 5,868,675. Accordingly, the Assignments of the U.S. 
Patent No, 5,868,675 are effective to the present reissue application. The duty of the 
inventors to execute further documents without further remuneration relating to 
reissues was transferred to Medtronic via the assignment chain enumerated above. 

Further, Medtronic will take a refusal by each inventor to execute the 
Supplemental Reissue Declaration based upon non-receipt of payment as simply a 
refusal to execute the Supplemental Reissue Declaration. Medtronic can overcome 
this refusal by filing a petition with the United States Patent and Trademark Office 
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under 37 CFR §1.47. Upon acceptance of such a Petition, inventor signatures are no 
longer necessary in the reissue application. 

Attached hereto (under Tab C) is a complete copy of the application as filed, the 
claims as pending, and the Supplemental Reissue Declaration. Medtronic against 
respectfully requests that each of the inventors execute the Supplemental Declaration 
and return it to Medtronic's Representative at Harness, Dickey & Pierce promptly via 
email at wamer(5).hdp.com or mail/courier to Harness Dickey, 5445 Corporate Dr, Suite 
200, Troy, Ml 48098. If the Supplemental Reissue Declarations are not received by 
November 14, 2011, Medtronic will assume that each of the inventors has refused to 
execute the Supplemental Reissue Declaration and Medtronic will continue prosecuting 
the reissue application based upon the refusal of each of the inventors to execute the 
Supplemental Reissue Declaration. 

Medtronic looks forward to your cooperation in this matter and requests that 
each inventor respond individually to this letter. 

Beginning of French Translation 

Le texte suivant est la traduction en francais de la lettre ci-dessus. 

Par la presente, Medtronic, Inc. (ci-apres Medtronic) demande 
respectueusement que chaque individu designe comme inventeur dans la demande de 
redelivrance U.S. 09/784,829, y compris Michel Scriban, Joel Henrion, Jean Francois 
UHL et Jean-Baptiste Thiebaut, signe la Declaration supplemental pour redelivrance 
envoyee precedemment et la renvoie a Richard W. Warner, representant de Medtronic 
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a Harness, Dickey & Pierce rapidement par email a rwarner@hdp.com ou par courrier 
a Harness Dickey, 5445 Corporate Dr, Suite 200, Troy, Ml 48098 avant le 14 
novernbre 2011. Pour votre commodite, nous vous envoyons des parquets prepayes 
a utiliser pour nous renvoyer la Declaration supplementaire pour redelivrance signee. 

Medtronic a examine votre lettre du 10 Octobre 2011 qui demande qu'un 
paiement de 70 800 euros (environ 98 743,68 $ americains) soit verse a tous les 
inventeurs pour leur intervention, y compris Texecution de la Declaration 
supplementaire pour redelivrance envoyee precedemment. Medtronic note qu'aucun 
paiement n f est requis en vertu de Taccord de cession qui a ete execute par tous les 
inventeurs le 22 juin 1992 et juge votre demande de paiement comme un refus de 
signer et de renvoyer la Declaration supplementaire pour redelivrance. Medtronic 
voudrait profiter de cette occasion pour expliquer en detail pourquoi aucun paiement 
n f est necessaire et offrir a chacun des inventeurs une ultime occasion de signer et de 
nous renvoyer la Declaration supplementaire pour redelivrance. 

Une cession a Diadix SA (sous Tonglet A), executee par chacun des inventeurs 
le 22 juin 1992, stipule : « Par la presente, les soussignes acceptent de transferer un 
tel interet a la demande dudit CESSIONNAIRE, ses ayants droit et representants 
legaux, et sans autre remuneration, dans et pour une ou toutes les divisions, les 
continuations, les substituts et les renditions de celui-ci ; et de certifier et d'executer 
tous les papiers pour le CESSIONNAIRE, ses ayants droit et representants juridiques, 
juges essentielles par le CESSIONNAIRE a !a protection du CONCESSIONNAIRE et 
de son titre de propriete et de Invention transferes par les presentes. » Ainsi, 
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Medtronic estime que I'execution de documents supplementaires pour Ies demandes 
de redeiivrance est un devoir de chaque inventeur a la demande de Medtronic et 
devrait etre accompli sans remuneration (paiement par exemple). 

Le brevet americain 5 868 675 d'origine a ete cede par Diadix SA a Deemed 
International (execute le 24 octobre 1995), puis par Deemed International a Elekta IGS 
S.A. (via un changement de nom execute le 14 aout 1997), et enfin par Elekta IGS SA 
a Medtronic, Inc (execute le 14 decembre 1999). Chacune de ces cessions est dument 
enregistree au «United States Patent and Trademark Office» (Bureau des brevets et 
des marques des Etats-Unis) dans le brevet americain N°. 5 868 675. La chame du 
titre, comme donnee par le «United States Patent and Trademark Office», est incluse 
sous Ponglet B et peut etre consultee a 
http://assignments.uspto.gov/assignments/q?db=pat&qt=pat&reel=&fram La 
demande en question, a laquelle la Declaration supplementaire pour redeiivrance est 
rattachee, est une rendition du brevet americain 5 868 675. En consequence, Ies 
cessions du brevet americain No. 5 868 675 restent en vigueur pour cette demande de 
redeiivrance. Le devoir des inventeurs de signer des documents supplementaires sans 
remuneration additionnelie concernant Ies renditions a ete transfere a Medtronic par la 
chaine de cessions ci-dessus enumeree. 

De plus, Medtronic considerera un refus par chaque inventeur d'executer la 
Declaration supplementaire pour redeiivrance basee sur la non-reception du paiement 
comme un simple refus d'executer la Declaration supplementaire pour redeiivrance. 
Medtronic peut surmonter ce refus en deposant une requete aupres du «United States 
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Patent and Trademark Office» sous Particle 37 CFR § 1.47. Des (Acceptation d'une 
telle petition, les signatures des inventeurs ne seront plus necessaires a la demande 
de redelivrance. 

Ci-joint (sous Fonglet C) est une copie complete de la demande deposee, les 
revendications en attente et la Declaration supplemental pour redelivrance. 
Medtronic demande a nouveau respectueusement que chacun des inventeurs execute 
la Supplemental Declaration et la renvoie au representant de Medtronic a Harness, 
Dickey & Pierce rapidement par email a warner@hdp.com ou par courrier a Harness, 
Dickey, 5445 Corporate Dr, Suite 200, Troy, MI 48098. Si les Declaration 
supplemental pour redelivrance ne sont pas regues avant le 14 novembre 2011, 
Medtronic supposera que chacun des inventeurs a refuse d'executer la Declaration 
supplementaire pour redelivrance et Medtronic continuera de poursuivre les demandes 
de redelivrance en utilisant le refus de chacun des inventeurs d'executer les 
Declaration supplementaire pour redelivrance. 
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Medtronic se rejouit de votre cooperation dans cette affaire et demande que 
chaque inventeur reponde individuellement a cette lettre. 



Best Regards, 




16367800.1 
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information in documents files in a patent application that may 
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information is included in documents submitted to the USPTO, 
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information from the documents before submitting them to the 
USPTO. 

Petitioner/application is advised that the record of a patent 
application is available to the public after publication of the 
application (unless a non-publication request in compliance with 37 
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credit card authorization forms PTO-2038 submitted for payment 
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the validity of the application or any patent issued thereon. 
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deposes a I'USPTO, les demandeurs / requerants devraient envisager de les 
enlever des documents avant de les soumettre a TUSPTO, 

Le requerant / demandeur est informe que le dossier de demande de brevet est a la 
disposition du public apres la publication de la demande (sauf si une demande de 
non-publication en conformite avec 37 CFR 1.213 (a) a ete faite dans cette 
demande) ou apres la deiivrance d'un brevet. En outre, le dossier d'une demande 
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referencee dans une application publiee ou un brevet delivre (voir 37 CFR 1.14). 
Les cheques et formulaires d'autorisation de carte de credit PTO-2038 soumis pour 
fins de paiement ne sont pas conserves dans le dossier de demande et ne sont 
done pas accessions au public. 
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ces declarations ont ete faites en sachant que toute fausse declaration volontaire 
est passible d'une amende ou d'une peine d'emprisonnement, ou les deux, en 
vertu de la loi 18 USC 1001 et que de telles declarations risquent de compromettre 
la validite de la demande ou d'un brevet delivre a partlr de celle-cl. 
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ABSTRACT 



An interactive system for a local intervention inside a region 
of a non-homogeneous structure, such as the skull of a 
patient, which is related to the frame of reference (R^ of an 
operation table, and which is connected to a reference 
structure comprising a plurality of base points. The system 
creates on a screen a representation of the non-homogeneous 
structure and of the reference structure connected thereto, 
provides the coordinates of the images of the base points in 
the first frame of reference (RJ, allows the marking of the 
coordinates of the base points in R 2 , and allows the carrying 
out of the local intervention with an active member such as 
a trephining tool, a needle, or a radioactive or chemical 
implant. The systems also optimizes the transfer of reference 
frames between R a and R 2 , from the coordinates of the base 
points in R 2 and the images in R^ by reducing down to a 
minimum the deviations between the coordinates of images 
in R x and the base points in R a after transfer. The system also 
establishes real time bi-directional coupling between: (1) an 
origin and a direction of intervention simulated on the 
screen, (2) the position of the active member. 
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INTERACTIVE SYSTEM FOR LOCAL 

INTERVENTION INSIDE A 
NONHUMOGENEOUS STRUCTURE 

The invention relates to an interactive system for local 5 
intervention inside a region of a nonhomogeneous structure. 

The performing of local interventions inside a nonho- 
mogeneous structure, such as intracranial surgical opera- 
tions or orthopedic surgery currently poses the problem of 
optimizing the intervention path or paths so as to secure, on 
the one hand, total intervention over the region or structure 
of interest, such as a turnor to be treated or explored and, on 
the other hand, minimal lesion lo the regions neighboring or 
adjoining the region of interest, this entailing the localizing 
and then the selecting of the regions of the nonhomogeneous 
structure which are least sensitive to being traversed or the 15 
least susceptible to damage as regards the integrity of the 
structure. 

Numerous works aimed at providing a solution to the 
abovementioned problem have hitherto been the subject of 
publications. Among the latter may be cited the article 20 
entitled "Three Dimensional Digitizer (Neuronavigator): 
New Equipment for computed Tomography Guided Stereo- 
taxic Surgery", published by Eiju Watanabe, M.D., Takashi 
Watanabe, M.D., Shinya Manaka, M.D., Yoshlaki 
Mayanagi, M.D., and Kintomo Takakura, M.D. Department 25 
of Neurosurgery, Faculty of Medicine, University of Tokyo, 
Japan, in the journal Surgery Neurol. 1987: 27 pp. 543-547, 
by Elsevier Science Publishing Co., Inc. The Patent WO-A- 
88 09151 teaches a similar item of equipment. 

In the abovementioned publications are described in 30 
particular a system and an operational mode on the basis of 
which a three-dimensional position marking system, of the 
probe type, makes it possible to mark the three-dimensional 
position coordinates of a nonhomogeneous structure, such as 
the head of a patient having to undergo a neurosurgical 35 
intervention, and then to put into correspondence as a 
function of the relative position of the nonhomoc.eneous 
structure a series of corresponding images consisting of 
two-dimensional images sectioned along an arbitrary 
direction, and obtained previously with the aid of a medical 40 
imaging method of the "scanner" type. 

The system and the operational mode mentioned above 
offer a sure advantage for the intervening surgeon since the 
latter has available, during the intervention, apart from a 
direct view of the intervention, at least one two-dimensional 45 
sectional view enabling him to be aware, in the sectional 
plane, of the state of performance of the intervention. 

However, and by virtue of the very design of the system 
and of the operational mode mentioned above, the latter 
allow neither a precise representation of the state of perfor- 50 
mance of the intervention, nor partially or totally automated 
conduct of the intervention in accordance with a program for 
advance of the instrument determined prior to the interven- 
tion. 

Such a system and such an operational mode cannot 55 
therefore claim to eradicate all man-made risk, since the 
intervention is still conducted by the surgeon alone. 

The objective of the present invention is to remedy the 
whole of the problem cited earlier, and in particular to 
propose a system permitting as exact as possible a 60 
correlation, at any instant, between an intervention modeling 
on the screen and the actual intervention, and furthermore 
the representation from one or more viewing angles, and if 
appropriate in one or more sectional planes, of the nonho- 
mogeneous structure, the sectional plane or planes possibly 65 
being for example perpendicular to the direction of the path 
of advance of the instrument or of the intervention tool. 
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Another objective of the present invention is also the 
implementation of a system permitting simulation of an 
optimal trajectory of advance of the tool, so as to constitute 
an assisted or fully programed intervention. 

Finally, an objective of the present invention is to pro- 
pose a system making it possible, on the basis of the 
simulated trajectory and of the programed intervention, to 
steer the movement of the instrument or tool to the said 
trajectory so as to carry out the programed intervention. 

The invention proposes to this effect an interactive 
system for local intervention inside a region of a nonhomo- 
geneous structure to which is tied a reference structure 
containing a plurality of base points, characterized in that it 
comprises: 

means of dynamic display by three-dimensional imaging 
of a representation of the nonhomogeneous structure 
and of a reference structure tied to the nonhomoge- 
neous structure, including images of the base points, 
means of delivering the coordinates of the images of the 

base points in the first reference frame, 
means of securing the position of the non-homogeneous 
structure and the reference structure with respect lo a 
second reference frame, 
marker means for delivering the coordinates of the base 

points in the second reference frame, 
means of intervention comprising an active member 
whose position is determined with respect to the second 
reference frame, 
means of optimizing the transfer of reference frames from 
the first reference frame to the second reference frame 
and vice versa, on the basis of the coordinates of the 
images of the base points in the first reference frame 
and of the coordinates of the base points in the second 
reference frame, in such a way as to reduce to a 
minimum the deviations between the coordinates of the 
images of the base points in the first reference frame 
and the coordinates of the base points, expressed in the 
said first reference frame with the aid of the said 
reference frame transfer tools, 
means for defining with respect to the first reference 
frame a simulated origin of intervention and a simu- 
lated direction of intervention, and 
reference frame transfer means using the said reference 
frame transfer tools to establish a bidirectional coupling 
between the simulated origin of intervention and the 
simulated direction of intervention and the position of 
the active member. 
A more detailed description of the system of the invention 
will be given below with reference to the drawings in which: 
FIG. 1 represents a general view of an interactive system 
for local intervention inside a region of a nonhomogeneous 
structure according to the present invention, 

FIG. 2 represents, in the case where the nonhomogeneous 
Structure consists of the head of a patient, and with a view 
to a neurosurgical intervention, a reference structure tied to 
the nonhomogeneous structure and enabling a correlation to 
be established between a "patient" reference frame and a 
reference frame of images of the patient which were made 
and stored previously, 

FIG. 3 represents an advantageous embodiment of the 
spacial distribution of the reference structure of FIG. 2, 

FIG. 4 preesents an advantageous embodiment of the 
intervention means set up on an operating table in the case 
of a neurosirgical intervention, 

FIGS. 5a and 5b represent a general flow diagram of 
functional steps implemented by the system, 
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FIGS. 6 thru 8 represent flow diagrams of programs 
permitting implementation of certain functional steps of 
FIG. Sb, 

FIG. 9a represents a flow diagram of a program permit- 
ting implementation of a functional step of FIG, 5*7, 5 

FIG. 9b represents a flow diagram of a program permit- 
ting implementation of another functional step of FIG. 5a, 

FIGS. 10a and 106 represent a general flow diagram of 
the successive steps of an interactive dialogue between the 
system of the present invention and the intervening surgeon 10 
and 

FIG. 10c represents a general flow diagram of the suc- 
cessive functional steps carried out by the system of the 
invention, having (sic) the intervention, prior to the 
intervention, during the intervention and after the interven- 15 
tion. 

The interactive system for local intervention according to 
the invention will firstly be described in connection with 
FIG. 1. 

Anonhomogeneous structure, denoted SNH, on which an 20 
intervention is to be performed, consists for example of the 
head of a patient in which a neurosurgical intervention is to 
be performed. It is however understood that the system of 
the invention can be used to carry out any type of interven- 
tion in any type of nonhomogeneous structure inside which 25 
structural and/or functional elements or units may be in 
evidence and whose integrity, during the intervention, is to 
be respected as far as possible. 

The system comprises means, denoted 1, of dynamic 
display by three-dimensional imaging, with respect to a first 30 
reference frame R a , of a representation (denoted RSR) of a 
reference structure SR (described later) tied to the structure 
SNH, and a representation or modeling of the nonhomoge- 
neous structure, denoted RSNH. 

More precisely, the means 1 make it possible to display a 35 
plurality of successive three-dimensional images, from dif- 
ferent angles, of the representations RSNH and RSR. 

The system of the invention also comprises means, 
denoted 2, of tied positioning, with respect to a second 
reference frame R 2 , of the structures SNH and SR. 40 

In the present non-limiting example, the head of the 
patient, bearing the reference structure SR, is fixed on an 
operating table TO to which are fixed the means 2 of tied 
positioning. 

Of course, the patient whose head has been placed in the 45 
means 2 for tied positioning has previously been subjected 
to the customary preparations, in order to enable him to 
undergo the intervention. 

The means 2 of the tied positioning with respect to R 2 will 
not be described in detail since they can consist of any 50 
means (such as a retaining headset) normally used in the 
field of surgery or neurosurgery. The reference frame R 2 can 
arbitrarily be defined as a tri -rectangular reference trihedron 
tied to the operating table TO, as represented in FIG. 1. 

Means 3 of marking, with respect to the second reference 55 
frame R 2 , the coordinates, denoted X2, Y2, Z2, of arbitrary 
points, and in particular of a certain number of base points 
of the reference structure SR are furthermore provided. 

These base points constituting the reference structure SR 
can consist of certain notable points and/or of marks fixed to 60 
the patient, at positions selected by the surgeon and in 
particular at these notable points. 

The system of the invention further comprises computing 
means 4 receiving means 3 of marking the coordinates X2, 
X2, 72. 65 

The computing means 4, as will be seen in detail later, are 
designed to elaborate optimal tools for reference frame 
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transfer using on the one hand the coordinates in R 2 , 
measured by the probe 3, of a plurality of base points of the 
structure SR, and on the other hand the coordinates in R l9 
determined by graphical tools of the computer M01 
(pointing by mouse, etc.), of the images of the corresponding 
base points in the representation RSR, so as to secure the 
best possible correlation between the information modeled 
in the computer equipment and the corresponding real-world 
information. 

There is furthermore provision for reference frame trans- 
fer means U designed to use the tools thus elaborated and 
to secure this correlation in real time. 

Moreover, means 40 are provided, as will be seen in detail 
later, for determining or modeling a reference origin of 
intervention ORI and a direction of intervention A. 

With the aid of the means 11, the modeled direction of 
intervention A can, at least prior to the intervention and at the 
start of the intervention, be materialized through an optical 
sighting system available to the surgeon, it being possible to 
steer this sighting system positionally with respect to the 
second reference frame It,. 

The sighting system will be described later. 

The system of the present invention finally comprises 
means 5 of intervention comprising an active member, 
denoted 50, whose position is specified with respect to the 
second reference frame R 2 . The active member can consist 
of the various tools used in surgical intervention. For 
example, in the case of an intercranial neurosurgical 
intervention, the active member could be a trephining tool, 
a needle, a laser or radioscope emission head, or an endo- 
scopic viewing system. 

According to an advantageous characteristic of the 
invention, by virtue of the reference frame transfer means 
11, the position of the active member can be controlled 
dynamically on the basis of the prior modeling of the origin 
of intervention ORI and of the direction of intervention A. 

The means 1 of dynamic display by three-dimensional 
imaging of the representations RSNH and RSR comprise a 
file 10 of two-dimensional image data. The file 10 consists 
for example of digitized data from tomographic sections, 
from radiographs, from maps of the patient's head, and 
contained in an appropriate mass memory. 

The successive tomographic sections can be produced 
prior to the intervention in a conventional manner, after the 
reference structure SR has been put in place on the nonho- 
mogeneous structure SNH. 

According to an advantageous feature, the reference 
structure SR can consist of a plurality of marks or notable 
points which can be both sensed by the marker means 3 and 
detected on the two-dimensional images obtained. 

Of course, the abovementioned two-dimensional tomo- 
graphic sections can likewise be produced by any medical 
imaging means such as a nuclear magnetic resonance sys- 
tem. 

In a characteristic and well-known manner, each two- 
dimensional image corresponding to a tomographic scanner 
section corresponds to a structural slice thickness of about 2 
to 3 mm, the pixels or image elements in the plane of the 
tomographic section being obtained with a precision of the 
order of ±1 mm, It is therefore understood that the marks or 
points constituting the reference structure SR appear on the 
images with a positional uncertainty, and an important 
feature of the invention will consist in minimizing these 
uncertainties as will be described later. 

The system also comprises first means 110 for calculating 
and reconstructing three-dimensional images from the data 
from the file 10. 
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It also comprises a high-resolution screen 12 permitting 
the displaying of one or more three-dimensional or two- 
dimensional images constituting so many representations of 
the reference structure RSR and of the nonhomogeneous 
structure SNH. 

Advantageously, the calculating means 110, the high- 
resolution screen and the mass memory containing the file 
10 form part of a computer of the workstation type with 
conventional design and denoted MOl. 

Preferably, the first calculating means 110 can consist of 
a CAD type program installed in the workstation MOl, 

By way of non-limiting example, this program can be 
derived from the software marketed under the tradename 
"AUTOCAD" by the "AUTODESK" company in the 
United States of America. 

Such software makes it possible, from the various digi- 
tized two-dimensional images, to reconstruct threedimen- 
sional images constituting the representations of the struc- 
tures RSR and RSNH in arbitrary orientations. 

Thus, as has furthermore been represented in FIG. 1, the 
calculating means 4 and 11 can consist of a third computer, 
denoted M02 in FIG. 1. 

The first and second computers MOl and M02 are 
interconnected "by a conventional digital link (bus or 
network). 

As a variant, the computers MOl and M02 can be 
replaced by a single workstation. 

The marker means 3 consist of a three-dimensional probe 
equipped with a tactile tip 30. 

This type of three-dimensional probe, known per se and 
not described in detail, consists of a plurality of hinged arms, 
marked in terms of position with respect to a base integral 
with the operating table TO. It makes it possible to ascertain 
the coordinates of the tactile tip 30 with respect to the origin- 
O;, of the reference frame R 2 with a precision better than 1 35 



The probe is for example equipped with resolvers deliv- 
ering signals representing the instantaneous position of the 
abovementioned tactile tip 30. The resolvers are themselves 
connected to the circuits for digital/analog conversion and 
sampling of the values representing these signals, these 
sampling circuits being interconnected in conventional man- 
ner to the second computer M02 in order to supply it with 
the coordinates X2, X2, Z2 of the tactile tip 30. 

As a variant or additionally, and as represented 
diagrammatically, the marker means 3 can comprise a set of 
video cameras 31 and 32 (or else infrared cameras) enabling 
pictures to be taken of the structures SNH and SR. 

The set of cameras can act as a stereoscopic system 
permitting the positional plotting of the base points of the 
reference structure SR, or of other points of the nonhomo- 
geneous structure SNH, with respect to the second reference 
frame R 2 . The positional plotting can be done for example 
by appending a laser beam emission system making it 
possible to illuminate successively the points whose coor- 
dinates are sought, appropriate software making it possible 
to then determine the position of these points one by one 
with respect to R 2 . This software will not be described since 
it can consist of position and shape recognition software 
normally available on the market. 

According to another variant, the marker means 3 can 
comprise a telemetry system. 

In this case, the marks of the structure SR can consist of 
small radiotransmitters implanted for example on the rel- 
evant points of the patient's head and designed to be visible 
on the two-dimensional images, appropriate electromagnetic 
or optical sensors (not shown) being provided in order to 



determine the coordinates of the said marks in the reference 
frame R 2 or in a reference frame tied to the latter. 

It is important to note here that the general function of the 
base points of the reference structure is, on the one hand, to 
be individually localizable on the reference structure, in 
order to deduce from this the coordinates in R 2 , and on the 
other hand, to be visualizable on the two-dimensional 
images so as to be identified (by their coordinates in R a ) and 
included in the representation RSR on the screen. 

It can therefore involve special marks affixed at arbitrary 
points of the lateral surface of the structure SNH, or else at 
notable points of the latter, or else, when the notable points 
can in themselves be localized with high precision both on 
the structure SNH and on the 2D sections, notable points 
totally devoid of marks. 

In FIG. 2 a plurality of marks, denoted Ml to Mi, these 
marks, in the case where the nonhomogeneous structure 
consists of the head of a patient, being localized for example 
between the eyebrows of the patient, on the latter's temples, 
and at the apex of the skull at a notable point such as the 
frontal median point. 

More generally, for a substantially ovoid volume consti- 
tuting the nonhomogeneous structure, there is advanta- 
geously provision for four base points at least on the outer 
surface of the volume. 

Thus, as has been represented in FIG, 3, the four marks 
Ml to M4 of the reference structure are distributed so as 
preferably to define a more or less symmetric tetrahedron. 
The symmetry of the tetrahedron, represented in FIG. 3, is 
materialized by the vertical symmetry plane PV and the 
horizontal symmetry plane PH. 

According to an advantageous characteristic, as will be 
seen later, the means of elaborating the reference frame 
transfer tools are designed to select three points of the 
tetrahedron which will define the "best plane" for the 
reference frame transfer. 

Also, the presence of four or more points enables the 
additional point(s) to validate a specified selection. 

More precisely, the presence of a minimum of four base 
40 points on the reference structure makes it possible to search 
for the minimum distortion between the points captured on 
the patient by the marker means consisting for example of 
the three-dimensional probe and the images of these points 
on the representation by three-dimensional imaging, the 
45 coordinates of which are calculated during processing. The 
best plane of the tetrahedron described earlier, that is to say 
the plane for which the uncertainty in the coordinates of the 
points between the points actually captured by the three- 
dimensional probe and the points of the representation of the 
reference structure RSR, is minimal, then becomes the 
reference plane for the reference frame transfer. Thus, the 
best correlation will be established between a modeled 
direction of intervention and a modeled origin of 
intervention, on the one hand, and the action of the member 
50. Preferably, the origin of intervention will be placed at the 
center of the region in which the intervention is to be carried 
out, that is to say a tumor observed or treated for example. 

Furthermore, it will be possible to take the noted residual 
uncertainty into account in order to effect the representation 
of the model and of the tools on the dynamic display means. 

A more detailed description of the means of intervention 
5 will now be given in connection with FIG. 4. 

Preferably, the means of intervention 5 comprise a car- 
riage 52 which is translationally mobile along the operating 
table TO, for example on a rack, denoted 54, whilst being 
driven by a motor, not shown, itself controlled by the 
computer M02 for example, via an appropriate link. This 
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movement system will not be described in detail since it appropriate, make it possible to record the path of the 
corresponds to a conventional movement system available sighting direction, constituting in particular the actual direc- 
on the market. As a variant, the carriage 52 can be mobile tion of intervention, on the representation of the nonhomo- 
ovcr a distinct path separated from the operating tabic TO, geneous structure in the dynamic display means 1. 
or immobile with respect to the operating table and then 5 Furthermore, as will be described later and in preferential 
constitute a support. manner, th& intervening surgeon will be able firstly to define 
The support carriage 52 comprises in the first place a a simulated intervention path and steer thereto the move- 
sighting member OV, constituting the above-mentioned ments of , the active member 50 in the nonhomogeneous 
sighting system, which can consist of a binocular telescope, 0nkr effectlvel y to secure aU or P art of the 

The sighting member OV enables the surgeon, prior to the 10 m T loa * - . . . . . _ n . 

L . . * ■ • iL . ^ i . lj 1,1 In this case, the progress or the intervention tool 50 is then 

actual intervention or during the latter, to sight the pre- steefed difect] ^ * t simulated fa (daU QRI A) fa 

sumed position of the region in which the intervention is to inyolving {h& / eference frame ^ u in ^ £ 

be earned out. express the path in the reference frame R 2 . 

Furthermore, and in a non-hmiting manner, with the A more detailed description of the implementation of the 

sighting member OV can be associated a helium-neon laser 15 operational mode of the system of the invention will now be 

emission system, denoted EL, maldng it possible to secure described in connection with FIGS. 5a and 5b. 

the aiming of a fine positioning or sighting laser beam on the According to FIG. 5a, the first step consists in obtaining 

structure SNH and in particular, as will be seen in detail and organizing in memory the two-dimensional image data 

later, to indicate to the surgeon the position of an entry point (step 100). Firstly, the nonhomogeneous structure SNH is 

PE prior to the intervention, to enable the latter to open the 20 prepared. In the case of a neurosurgical intervention for 

skull at the appropriate location, and likewise to indicate to example, this means that the patient's head can be equipped 

him what the direction of intervention will be. Additionally, with marks constituting the base points of the reference 

the illuminating of the relevant point of the nonhomoge- structure SR. These marks can be produced by means of 

neous structure or at the very least the lateral surface of the points consisting of a dye partially absorbing the X-rays, 

latter enables the video cameras 31 and 32 to carry out, if 25 such as a radiopaque dye. 

necessary, a positional plotting. The abovementioned marks are implanted by the surgeon 

Preferably, a system for measuring position by telemetry on the patient's head at notable points of the latter [sic], and 

53 is provided to secure the precise measurement of the images can then be taken of the nonhomogeneous structure 

position of the support carriage 52 of the sighting member SNH by tomography for example, by means of an apparatus 

OV and of the laser emission system EL. During the 30 of the X-ray scanner type. 

operation, and in order to secure the intervention, the This operation will not be described in detail since it 

carriage 52 can be moved along the rack 54, the position of corresponds to conventional operations in the field of medi- 

the carriage 52 being measured very precisely by means of cal imaging. 

the system 53. The telemetry system 53 is interconnected The two-dimensional image data obtained are then con- 

with the microcomputer M02 by an appropriate link. 35 stituted as digitized data in the file 10, these data being 

The means of intervention 5 can advantageously consist themselves marked with respect to the reference frame R 1 

of a guide arm 51 for the active member 50. . and making it possible, on demand, to restore the two- 

The guide arm 51 can advantageously consist of several dimensional images onto the dynamic display means 1, 

hinged segments, each hinge being equipped with motors these images representing superimposed sections of the 

and resolvers making it possible to secure control of move- 40 nonhomogeneous structure SNH. 

ment of the end of the support arm and the positional From the digitized image data available to the surgeon, 

plotting of this same end and therefore of the active member the latter then proceeds, as indicated at 101 in FIG. 5a, to 

50 according to six degrees of freedom with respect to the select the structures of interest of the abovementioned 

carriage 52. The six degrees of freedom comprise, of course, images. 

three translational degrees of freedom with respect to a 45 The purpose of this step is to facilitate the work of the 
reference frame tied to the carriage 52 and three rotational surgeon by forming three-dimensional images which con- 
degrees of freedom along these same axes. tain only the contours of the elements of the structure which 

Thus, the support arm 51 and the member 50 are marked are essential for geometrical definition and real-time moni- 

in terms of instantaneous position with respect to the second toring of the progress of the intervention, 

reference frame R 2 , on the one hand by way of the positional 50 In the case where the nonhomogeneous structure SNH 

plot of the mobile carriage 52 and, on the other hand, by way consists of the head of a patient, an analysis of the two- 

of the resolvers associated with each hinge of the support dimensional image data makes it possible, from values of 

arm 51. optical density of the corresponding image-points, straight- 

In the case of an intracranial neurosurgical surgical away to extract the contours of the skull, to check the 

intervention, the active member 50 can be removed and can 55 distance scales, etc. 

consist of a trephining tool, a needle or radioactive or Preferably, the abovementioned operations are performed 

chemical implant, a laser or radioisotope emission head or on a rectangle of interest for a given two-dimensional image, 

an endoscopic viewing system. These various members will this making it possible, by moving the rectangle of interest, 

not be described since they correspond to instruments nor- to cover the whole of the image. 

mally used in neurosurgery. 60 The above analysis is performed by means of suitable 

The materializing of the modeled direction of intervention software which thus makes it possible to extract and vec- 

can be effective by means of the laser emitter EL. This torize the contours of the structures which will be modeled 

sighting being performed, the guide arm 51 can then be in the representations RSNH and RSR. 

brought manually or in steered manner into superposition The structures modeled in the case of a neurosurgical 

with the direction of intervention A. 65 intervention are for example the skull, the cerebral 

In the case of manual positioning, the resolvers associated ventricles, the tumor to be observed or treated, the falx 

with the sighting member OV and the laser emitter EL, if cerebri, and the various functional regions. 
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According to a feature of the interactive system of the 
invention, the surgeon may have available a digitizing table 
or other graphics peripheral making it possible, for each 
displayed two-dimensional image, to rectify or complete the 
definition of the contour of a particular region of interest. 

It will be noted finally that by superimposing the extracted 
contours on the displayed two-dimensional image, the sur- 
geon will be able to validate the extractions carried out. 

The extracted contours are next processed by sampling 
points to obtain their coordinates in the reference frame R a , 
it being possible to constitute these coordinates as an ASCII 
type file. This involves step 102 for generating the three- 
dimensional data base. 

This step is followed by a step 103 of reconstructing the 
three-dimensional model. This step consists firstly, with the 
aid of the CAD type software, in carrying out on the basis 
of the contours of the structures of interest constituted as 
vectorized two-dimensional images an extrapolation 
between the various sectional planes. 

The abovementioned extrapolation is carried out prefer- 
ably by means of a "B-spline" type algorithm which seems 
best suited. This extrapolation transforms a discrete item of 
information, namely the successive sections obtained by 
means of the scanner analysis, into a continuous model 
permitting three-dimensional representation of the volume 
envelopes of the structures. 

It should be noted that the reconstruction of the volumes 
constituting the. structures of interest introduces an approxi- 
mation related in particular to the spacing and non-zero 
thickness of the acquisition sections. An important charac- 
teristic of the invention, as explained in detail elsewhere, is 
on the one hand to minimize the resulting uncertainties in the 
patient-model correlation, and on the other hand to take into 
account the residual uncertainties. 
. The CAD type software used possesses standard functions 
which enable the model to be manipulated in space by 
displaying it from different viewpoints through just a crite- 
rion defined by the surgeon (step 104). 

The software can also reconstruct sectional representation 
planes of the nonhomogeneous structure which differ from 
the planes of the images from the file 10, this making it 
possible in particular to develop knowledge enhancing the 
data for the representation by building up a neuro- 
anatomical map. 

The surgeon can next (step 105) determine a model of 
intervention strategy taking into account the modeled struc- 
tures of interest, by evaluating the distance and angle ratios 
on the two-and three-dimensional representations displayed. 

This intervention strategy will consist, in actual fact, on 
the one hand in localizing the tumor and in associating 
therewith a "target point", which will subsequently be able 
to substitute for the origin common to all the objects (real 
and images) treated by the system, and on the other hand in 
determining a simulated intervention path respecting to the 
maximum the integrity of the structures of interest. This step 
can be carried out "in the office", involving only the work- 
station. 

Once this operation is performed and prior to the 
intervention, the following phase consists in implementing 
the steps required to establish as exact as possible a corre- 
lation between the structure SNH (real world) and the 
representation RSNH (computer world). This involves steps 
106 to 109 of FIG. $b. 

Firstly, as represented in FIG. Sb at step 107, marking of 
the base points of the reference structure SR with respect to 
the second reference frame is carried out with the aid of the 
marker means 3, by delivering to the system the coordinates 
X2, Y2, Z2 of the said base points. 
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The following step 106 consists in identifying on the 
representations RSNH and RSR displayed on the screen the 
images of the base points which have just been marked. 
More precisely, with the aid of appropriate graphics 

5 peripherals, these representations (images) of the base points 
are selected one by one, the workstation supplying on each 
occasion (in this instance to the computer M02) the coor- 
dinates of these points represented in the reference frame R a . 
Thus the computer M02 has available a first set of 

10 three-dimensional coordinates representing the position of 
the base points in R 2 , and a second set of three-dimensional 
coordinates representing the position of the representations 
of the base points in R 3 . 

According to an essential feature of the invention, these 

15 data will be used to elaborate at 108, 109, tools for reference 
frame transfer (from R a to R 2 and vice versa) by calling 
upon an intermediate reference frame determined from the 
base points and constituting an intermediate reference frame 
specific to the reconstructed model. 

20 More precisely, the intermediate reference frame is con- 
structed from three base points selected so that, in this 
reference frame, the coordinates of the other base points 
after transfer from R 2 and the coordinates of the represen- 
tations of these other base points after transfer from R a are 

25 expressed with the greatest consistency and minimum dis- 
tortion. 

When the step of elaborating the reference frame transfer 
tools is concluded, these tools can be used by he system to 
secure optimal coupling between the real world and the 

30 computer world (step 1110). 

Furthermore, according to a subsidiary feature of the 
present invention, the system can create on the display 
means a representation of the nonhomogeneous structure 
and of the intervention member which takes account of the 

35 deviations and distortions remaining after the "best" refer- 
ence frame transfer tools have been selected (residual 
uncertainties). More precisely, from these deviations can be 
deduced by the calculating means a standard error likely to 
appear in the mutual positioning between the representation 

40 of the nonhomogeneous structure and the representation of 
elements (tools, sighting axes, etc.) referenced on R 2 when 
using the reference frame transfer tools. This residual 
uncertainty, which may in practice be given substance 
through an error matrix, can be used for example to repre- 

45 sent certain contours (tool, structures of interest to be 
avoided during the intervention, etc.) with dimensions larger 
than those which would normally be represented starting 
from the three-dimensional data base or with the aid of 
coordinates marked in R 2 , the said larger dimensions being 

50 deduced from the "normal" dimensions by involving the 
error matrix. For example, if the member were represented 
normally, in transverse section, by a circle of diameter Dl, 
a circle of diameter D2>D1 can be represented in substance, 
with the difference D2-D1 deduced from the standard error 

55 value. In this way, when a direction of intervention will be 
selected making it possible to avoid traversing certain struc- 
tures of interest, the taking into account of an "enlarged" size 
of the intervention tool will eradicate any risk of the 
member, because of the abovementioned errors, accidently 

60 traversing these structures. 

Back at step 105, and as will be seen in more detail with 
reference to FIGS. 9a and 96, the reference origin of 
intervention ORI and the direction of intervention A, that is 
to say the simulated intervention path, can be determined 

65 according to various procedures. 

According to a first procedure, the trajectory can be 
defined from two points, namely an entry point PE (FIG. 3) 
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and a target point, that is to say substantially the center of the 
structure of interest consisting of the tumor to be observed 
or treated. Initially, these two points are localized on the 
model represented on the screen. 

According to a second methodology, the trajectory can be 5 
determined from the abovementioned target point and from 
a direction which takes account of the Lypes of structures of 
interest and of their positions with a view to optimally 
respecting their integrity. 

After the abovementioned step 108, the surgeon can at 
step 1110 perform the actual intervention. 

The intervention can advantageously be performed by 
steering the tool or active member over the simulated 
intervention path, determined in step U10. 

As a variant, given that the support arm 51 for the active 
member, equipped with its resolvers, continuously delivers 15 
the coordinates in R 2 of the said active member to the 
sysLera, it is also possible to perform the operation manually 
or semi-manually, by monitoring on the screen the position 
and motions of a representation of the tool and by comparing 
them with the simulated, displayed intervention path. 20 

It will furthermore be noted that the modeled direction of 
intervention can be materialized with the aid of the laser 
beam described earlier, the positioning of the latter (with 
respect to R 2 ) being likewise carried out by virtue of the 
reference frame transfer tools. 25 

Certain functional features of the system of the invention 
will now be described in further detail with reference to 
FIGS. 6, 7, 8, 9a and 9b. 

The module for elaborating the reference frame transfer 
tools (steps 108, 109 of FIG. 5b) will firstly be described 30 
with reference to FIG. 6. 

This module comprises a first sub-module 1001 for 
acquiring three points A, B, C, the images of the base points 
of SR on the representation RSNH (the coordinates of these 
points being expressed in the computer reference frame R x ) f 35 
by successive selections of these points on the representa- 
tion. To this effect, the surgeon is led, by means of a graphics 
interface such as a "mouse" to point successively at the three 
selected points A, B, C 

The module for preparing the transfer tools also com- 40 
prises a second sub-module, denoted 1002, for creating a 
unit three-dimensional orthogonal matrix M, this matrix 
being characteristic of a right-handed orthonormal basis 

represented by three unit vectors i , j , k , which define an 
intermediate reference frame tied to R v 45 

The unit vectors i , j and k* are given by the relations: 



The sub-module 1003 is itself followed, as represented in 
FIG. 6, by a fourth sub-module 1004 for creating a unit 
three-dimensional orthogonal matrix N, characteristic of a 
right-handed orthonormal basis comprising three unit vec- 
tors i f , j ' k' and which is tied to the second reference 
frame R 2 owing to the fact that the nonhomogeneous struc- 
ture SNH is positionally tied with respect to this reference 
frame. 

The three unit vectors i j ', k ' are defined by the 
relations: 
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where || || designates the norm of the relevant vector, 

In the above relations, the sign "A" designates the vector 
product of the relevant vectors. 60 

Similarly, the module for preparing the transfer tools 
comprises a third sub-module, denoted 1003, for acquiring 
three base points D, E, F, of the structure SR, these three 
points being those whose images on the model are the points 
A, B, C respectively. For this purpose, the surgeon, for 65 
example by means of the tactile tip 30, successively senses 
these three points to obtain their coordinates in R 2 . 
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As indicated above, to the extent that the base points of 
the reference structure can be marked in R 2 with high 
precision, so their representation in the computer base R 2 is 
marked with a certain margin of error given on the one hand 
the non-zero thickness (typically from 2 to 3 ram) of the 
slices represented by the two-dimensional images from the 
file 10, and on the other hand (in general to a lesser extent) 
the definition of each image element or pixel of a section. 

According to the invention, once a pair of transfer matri- 
ces M, N has been elaborated with selected points A, B, C, 
D, E, F, it is sought to validate this selection by using one or 
more additional base points; more precisely, for the or each 
additional base point, this point is marked in R 2 with the aid 
of the probe 30, the representation of this point is marked in 
Rj after selection on the screen, and then the matrices N and 
M are applied respectively to the coordinates obtained, in 

order to obtain their expressions in the bases ( i j k ') and 

( i , j , k) respectively. If these expressions are in good 
agreement, these two bases can be regarded as a single 
intermediate reference frame, this securing the exact as 
possible mathematical coupling between the computer ref- 
erence frame K t tied to the model and the "real" reference 
frame R 2 tied to the patient. 

In practice, the module for elaborating the reference frame 
transfer tools can be designed to perform steps 1001 to 1004 
in succession on basic triples which differ on each occasion 
(for example, if four base points have been defined associ- 
ated with four representations in RSR, there are four pos- 
sible triples), in order to perform the validation step 1005 for 
each of these selections and finally in order to choose the 
triple for which the best validation is obtained, that is to say 
for which the deviation between the abovementioned 
expressions is smallest. This triple defines the "best plane" 
mentioned elsewhere in the description, and results in the 
"best" transfer matrices M and N. 

As a variant, it will be possible for the selection of the best 
plane to be made at least in part by the surgeon by virtue of 
his experience. 

It should be noted that the reference frame transfer will 
only be concluded by supplementing the matrix calculation 
with the matrices M, N with a transfer of origin, so as to 
create a new common origin for example at the center of the 
tumor to be observed or treated (point ORI). This transfer of 
origin is effected simply by appropriate subtraction of vec- 
tors on the one hand on the coordinates in R 2 , and on the 
other hand on the coordinates in R 2 . These vectors to be 
subtracted are determined after localization of the center of 
the tumor on the representation. 
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Furthermore, the means described above for establishing 
the coupling between the patient's world and the model's 
world can also be used to couple to the model's world that 
of map data, also stored in the workstation and expressed in 
a different reference frame denoted R 3 . In this case, since 5 
these data contain no specific visible made, the earlier 
described elaboration of matrices is performed by substitut- 
ing for these marks the positions of notable points of the 
patient's head. These may be temporal points, the frontal 
median point, the apex of the skull, the center of gravity of 
the orbits of the eyes, etc. 

The corresponding points of the model can be obtained 
either by selection by mouse or graphics tablet on the model, 
or by sensing on the patient himself and then using the 
transfer matrices. 

The above step of elaborating the reference frame transfer 15 
tools, conducted in practice by the calculating means 4, 
makes it possible subsequently to implement the reference 
frame transfer means (FIGS. 7 and 8). 

With reference to FIG. 7, the first transfer sub-module 201 
comprises a procedure denoted 2010 for aquiring the coor- 20 
dinates XM, YM, ZM, expressed in R A , of the point to be 
transferred, by selecting on the representation. 

The procedure 2010 followed by a procedure 2011 for 
calculating the coordinates XP, YP, ZP (expressed in R 2 ) of 
the corresponding real point on the patient through the 2 s 
transformation: 

{XP, YP, ZP}*M*N- U {XM, YM, ZM} where M * N~* 
represents the product of the matrix M and the inverse 
matrix N. 

The procedure 2011 is followed by a processing proce- 30 
dure 2012 utilizing the calculated coordinates XP, YP, ZP, 
for example to indicate the corresponding point on the 
surface of the structure SNH by means of the laser emission 
system EL, or again to secure the intervention at the relevant 
point with coordinates XP, YP, ZP (by steering the active 35 
member). 

Conversely, in order to secure a transfer from SNH to 
RSNH, the second sub-module 202 comprises (FIG. 8) a 
procedure denoted 2020 for acquiring on the structure SNH 
the coordinates XP, YP, ZP (expressed in R 2 ) of a point to be 40 
transferred. 

These coordinates can be obtained by means of the tactile 
tip 30 for example. The procedure 2020 is followed by a 
procedure 2021 for calculating the corresponding coordi- 
nates XM, YM, ZM in R 2 through the transformation: 45 

{XM, YM, 2M}^W X *{XP 9 YP, ZP} 

A procedure 2022 next makes it possible to effect the 
displaying of the point with coordinates XM, YM, ZM on 
the model or again of a straight line or of a plane passing 
through this point and furthermore meeting other criteria. 50 

It will be noted here that the two sub-modules 201, 202 
can used [sic] by the surgeon at any moment for the purpose 
of checking the valid nature of the transfer tools; in 
particular, it is possible to check at any time that a real base 
point, with coordinates known both in R 2 and Rj (for 55 
example a base point of SR or an arbitrary notable point of 
the structure SNH visible on the images), correctly relocates 
with respect to its image after transferring the coordinates in 
step 2011. 

In the event of an excessive difference, a new step of 60 
elaboration of the transfer tools is performed. 

Furthermore, the sub-modules 201, 202 can be designed 
to also integrate the taking into account of the residual 
uncertainty, as spoken of above, so as for example to 
represent on the screen a point sensed not in a pointwise 65 
manner, but in the form for example of a circle or a sphere 
representing the said uncertainty. 
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From a simulated intervention path, for example on the 
representation RSNH, or from any other straight line 
selected by the surgeon, the invention furthermore enables 
the model to be represented on the screen from a viewpoint 
corresponding to this straight line. Thus the third transfer 
subroutine comprises, as represented in FIGS. 9a and 9b, a 
first module 301 for visualizing the representation in a 
direction given by two points and a second module 302 for 
visualizing the representation in a direction given by an 
angle of elevation and an angle of azimuth. 

The first module 301 for visualizing the representation in 
a direction given by two points comprises a first sub-module 
denoted 3010 permitting acquisition of the two relevant 
points which will define the selected direction. The coordi- 
nates of these points are expressed in the reference frame R lf 
these points having either been acquired previously on the 
nonhomogeneous structure SNH for example by means of 
the tactile tip 30 and then subjected to the reference frame 
transfer, or chosen directly on the representation by means 
of the graphics interface of the "mouse" type. 

The first sub-module 3010 is followed by a second 
sub-module denoted 3011 permitting the creation of a unit, 
orthogonal three-dimensional matrix V characteristic of a 

right-handed orthonormal basis i ,r , j k " the unit vectors 

i j k M , being determined through the relations: 

r -AB/\\AB ||; 
7" • r » oJ u * 7' « 0;\\ ? 11 « 1; 

where "A" represents the vector product and "." symbolizes 
the scalar product. 

The sub-module 3011 is followed by a routine 3012 
making it possible to secure for all the points of the entities 
(structures of interest) of the three-dimensional data base of 
coordinates XW, YW, ZW in R a a conversion into the 

orthonormal basis (7 M , X"> £") by me relation: 

{XV, YV, ZV}=V*{XW, YW, ZW} 

The subroutine 3013 is then followed by a subroutine 
3014 for displaying the plane i", j ,r , the subroutines 3013 and 
3014 being called up for all the points, as symbolized by the 
arrow returning to block 3012 in FIG. 9a. 

When all the points have been processed, an output 
module 3015 permits return to a general module, which will 
be described later in the description. It is understood that this 
module enables two-dimensional images to be reconstructed 
in planes perpendicular to the direction defined by A and B. 

In the same way, the second module 302 (FIG. 9b) for 
visualizing the representation from a viewpoint given by an 
angle of elevation and an angle of azimuth comprises a first 
sub-module 3020 for acquiring the two angles in the repre- 
sentation frame of reference. 

The selection of the angles of elevation and of azimuth 
can be made by selecting from a predefined data base or by 
moving software cursers associated with each view or else 
by modification relative to a current direction, such as the 
modeled direction of intervention. The sub-module 3020 is 
itself followed by a second sub-module 3021 for creating a 
unit orthoganal three-dimensional matrix W characteristic of 

a right-handed orthonormal basis of unit vectors i m , j '", 

k ,n . They are defined by the relations: 
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7"' = 0; 

k " • z "' = sin(azimuth) 

i n * > y => cos (elevation); 
i *" ■ i"' H sin (elevation) 
7"' - ~ft "* A T"' 

A routine 3022 is then called for all the points of the 
entities of the three-dimensional data base of coordinates 
XW, YW, ZW and enables a first sub-routine 3023 to be 
called permitting calculation of the coordinates of the rel- 
evant point in the right-handed orthonormal bases i '", 

X m k ,u through the transformation: 

{XV, YV, ZV}-V*{XW, YW, ZW} 

The sub-routine 3023 is itself followed by a sub-routine 
3024 for displaying the plane i'", j f ", the two sub-routines 
3023 and 3024 then being called up for each point as 
symbolized by the return via the arrow to the block 3022 for 
calling the abovementioned routine. When all the points 
have been processed, an output sub-module 3025 permits a 
return to the general menu. 

Of course, all of the programs, sub-routines, modules, 
sub-modules and routines destroyed earlier are managed by 
a general "menu" type program so as to permit interactive 
driving of the system by screen dialogue with the interven- 
ing surgeon by specific screen pages. 

A more specific description of a general flow diagram 
illustrating this general program will now be given in 
connection with FIGS. 10a and 106. 

Thus, in FIG. 10a has been represented in succession a 
screen page 4000 relating to the loading of data from the 
digitized file 10, followed by a screen page 4001 making it 
possible to secure the parameterizing of the grey scales of 
the display on the dynamic display means 1 and to calibrate 
the image, for example. 

The screen page 4001 is followed by a screen page 4002 
making it possible to effect the generation of a global view 
and then a step or screen page 4003 makes it possible to 
effect an automatic distribution of the sections on the screen 
of the workstation. 

A screen page 4004 makes it possible to effect a manual 
selection of sections and then a screen page 4005 makes it 
possible to effect the selection of the strategy (search for the 
entry points and for the possible directions of intervention, 
first localizing of the target (tumor . . . ) to be treated . . . ), 
as defined earlier, and to select the position and horizontal, 
sagittal and frontal distribution of the sections. 

A screen page 4006 also makes it possible to effect a 
display of the settings of a possible stereotaxic frame. 

It will be recalled that the reference structure SR advan- 
tageously replaces the stereotaxic frame formerly used to 
effect the marking of position inside the patient's skull 

There may furthermore be provided a screen-page 4007 
for choosing strategic sections by three-dimensional 
viewing, on selection by the surgeon, and then at 4008 the 
aligning of the references of the peripherals (tool, sighting 
members, etc., with the aid of the probe 30. 

A screen page 4009 is also provided to effect the search 
for the base points on the patient with the aid of the said 
probe, following which the steps of construction of the 
reference frame transfer tools and of actual reference frame 
transfer are performed, preferably in a user-transparent 
manner. 
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Another screen page 4010 is then provided, so as to effect 
the localizing of the target oh the representation (for 
example a tumor to be observed or treated in the case of a 
neurosurgical intervention) in order subsequently to deter- 
5 mine a simulated intervention path. 

Then a new screen page 4011 makes it possible to effect 
the setting of the guides for the tool on the basis of this 
simulated path before opening up the skin and bone flaps on 
the patient's skull. 
■ TTien a new localizing step 4012 makes it possible to 
check whether the position of the guides corresponds cor- 
rectly to the simulated intervention path. 

The screen page 4012 is followed by a so-called inter- 
vention screen page, the intervention being performed in 
accordance with step 1110 of FIG. 5b. 
15 A more detailed description of the interactive dialogue 
between the surgeon and the system during a surgical, and 
in particular a neurosurgical, intervention will follow with 
reference to FIG. 10c and to all of the preceding description. 

The steps of FIG. 10c are also integrated in the general 
20 program mentioned earlier; there are undertaken in succes- 
sion a first phase I (preparation of the intervention), then a 
second phase II, (prior to the actual intervention, the patient 
is placed in a condition for intervention, the reference 
structure SR being tied to the second reference frame RJ, 
25 then a third phase III (intervention) and finally a post- 
intervention phase IV. 

With a view to preparing the intervention, the system 
requests the surgeon (step 5000) to choose the elementary 
structures of interest (for example bones of the skull, 
ventricles, vascular regions, the tumor to be explored or 
treated, and the images of the marks constituting in the first 
reference frame the representation RSR). 

The choice of the elementary structures of interest is made 
on the display of the tomographic images, for example, 
called up from the digitized file 10. 
35 The system next performs, at step 5001, a modeling of the 
structures of interest, as described earlier. Then, the nonho- 
mogeneous structure having been thus constituted as a 
three-dimensional model RSNH displayed on the screen, the 
intervening surgeon is then led to perform a simulation by 
40 three-dimensional imaging, at step 5002, with a view to 
defining the intervention path of the tool 50. 

During phase II the patient being placed in a condition for 
intervention and his head and the reference structure SR 
being tied to the second reference frame R 2 , the surgeon 
45 performs at step 5003 a search for the position of the marks 
Ml to M4 constituting base points of the reference structure 
in the second reference frame R 2 , and then during a step 
5004, performs a search for the position of the sighting 
systems, visualizing member OY, or of the tools and inter- 
50 vention instruments 50, still in the second reference frame 
R 2 , so as, if appropriate, to align these implements with 
respect to R 2 . 

The system then performs the validation of the 
intervention/patient spaces and representation by three- 

55 dimensional imaging in order to determine next the common 
origin of intervention ORI. In other words, the matrix 
reference frame transfer described above is supplemented 
with the necessary origin translations (origins 01 and 02 
aligned on ORI). 

60 This operation is performed as described earlier. 

Phase III corresponds to the intervention, during which 
the system effects at step 5006 a permanent coupling in real 
time between the direction of aim of the active member 50, 
and/or of the direction of aim of the sighting member OV 

65 (and if appropriate of the laser beam), with the direction of 
aim (of observation) simulated by three-dimensional imag- 
ing on the display means 1, and vice versa. 



5,868 ; 

17 

In the following step 5007, the coupling is effected of the 
movements and motions of the intervention instrument with 
their movements simulated by three-dimensional imaging, 
with automatic or manual conduct of the intervention. 

As noted at 5008, the surgeon can be supplied with a 5 
permanent display of the original two-dimensional sectional 
images in planes specified with respect to the origin ORI and 
to the direction of intervention. Such a display enables the 
surgeon at any time to follow the progress of the intervention 
in real time and to be assured that the intervention is 10 
proceeding in accordance with the simulated intervention. In 
phase IV which is executed after the intervention, the system 
effects a saving of the data acquired during the intervention, 
this saving making it possible subsequently to effect a 
comparison in real time or deferred in the event of succes- is 
sive interventions on the same patient. 

Furthermore, the saved data make it possible to effect a 
playback of the operations carried out with the option of 
detailing and supplementing the regions traversed by the 
active member 50. 20 

Thus, a particularly powerful interactive system for local 
intervention has been described. 

Thus, the system which is the subject of the present 1 
invention makes it possible to represent a model containing 
only the essential structures of the nonhomogeneous 25 
structure, this facilitating the work of preparation and of 
monitoring of the intervention by the surgeon. 

Moreover, the system, by virtue of the algorithms used 
and in particular by minimizing the distortion between the 
real base points and their images in the 2D sections or the 30 
maps, makes it possible to establish a two-way coupling 
between the real world and the computer world through 
which the transfer errors are minimized, making possible 
concrete exploitation of the imaging data in order to steer the 
intervention tool. 35 

To summarize, the system makes possible an ineractive 
[sic] medical usage not only to create a three-dimensional 
model of the nonhomogeneous structure but also to permit 
a marking in real time with respect to the internal structures 
and to guide the surgeon in the intervention phase. 40 

More generally, the invention makes it possible to end up 
with a coherent system in respect of: 

the two-dimensional imaging data (scanner sections, 
maps, etc.) 

the three-dimensional data base; 45 
the data supplied by the marker means 3 in the reference 
frame R 2 ; 

the coordinate data for the sighting systems and interven- 
tion tools; 5Q 

the real world of the patient on the operating table. 

Accordingly, the options offered by the system are, in a 
non-limiting manner, the following: 

the tools and of [sic] their position can be represented on 
the screen; 55 

the position of a point on the screen can be materialized 
on the patient for example with the aid of the laser 
emission device EL; 

the orientation and the path of a tool such as a needle can 
be represented on the screen and materialized on the eo 
patient optically (laser emission) or mechanically 
(positioning of the guide-arm in which the tool is 
guided in translation): 

an image of the patient, yielded for example by a system 
for taking pictures if appropriate in relief, can be 65 
superimposed on the three-dimensional representation 
modeled on the screen; thus, any change in the soft 
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external parts of the patient can be visualized as com- 
pared with the capture by the scanner; 

it being possible for the surgeon's field of view given by 
a sighting member (such as a surgical microscope) to be 
referenced with respect to R 2 , the direction of visual- 
ization of the model on the screen can be made identical 
to the real sight by the sighting member; 

finally, the three-dimensional images, normally displayed 
on the screen in the preceding description, may as a 
variant be introduced into the surgeon's microscope so 
as to obtain the superposition of the real image and the 
representation of the model. 

We claim: 

1. An interactive system for local intervention inside a 
region of a non-homogeneous structure to which is con- 
nected a reference structure containing a plurality of base 
points, the interactive system comprising: 

means for dynamically displaying a three-dimensional 
image of a representation of the non-homogeneous 
structure and of the reference structure connected to the 
non-homogeneous structure, wherein the three- 
dimensional image also includes a plurality of images 
of the plurality of base points; 

means for determining a set of coordinates of the plurality 
of images of the plurality of base points in a first 
reference frame; 

means for fixing a position of the non-homogeneous 
structure and of the reference structure with respect to 
a second reference frame; 

means for determining a set of coordinates of the plurality 
of base points in the second reference frame; 

means of intervention comprising an active member 
whose position is determined with respect to the second 
reference frame; 

means for generating a plurality of reference frame trans- 
lation tools for translating a plurality of reference 
frames from the first reference frame to the second 
reference frame and vice versa, based on the set of 
coordinates of the plurality of images of the plurality of 
base points in the first reference frame and of the set of 
coordinates of the plurality of base points in the second 
reference frame, in such a way as to reduce to a 
minimum at least one of a set of deviations between the 
set of coordinates of the plurality of images of the 
plurality of base points in the first reference frame and 
the set of coordinates of the base points, expressed in 
the first reference frame using the plurality of reference 
frame translation tools; 

means for defining, with respect to the first reference 
frame, a simulated origin of intervention and a simu- 
lated direction of intervention; and, 

means for transferring the plurality of reference frames 
using the plurality of reference frame translation tools 
to establish a bidirectional coupling between the simu- 
lated origin of intervention and the simulated direction 
of intervention and the position of the active member. 

2. The interactive system according to claim 1, wherein 
the plurality of reference frame translation tools comprise: 

means for creating a matrix (M) for transferring between 
the first reference frame and a first intermediate refer- 
ence frame based on a set of coordinates of a set of 
three images of a set of three base points of the 
reference structure; 

means for creating a matrix (N) for transferring between 
the second reference frame and a second intermediate 
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reference frame based on the set of coordinates of the 
set of three images of the set of three base points of the 
reference structure; and, 
means for validating matrix (M) and matrix (N) based on 
the set of three base points and the set of three images, 5 
such that at least one deviation between an expression 
for at least one additional base point in the second 
intermediate reference frame and an expression for at 
least one image point of the additional base point in the 
first intermediate reference frame is reduced to a mini- 10 
mum. 

3. The interactive system according to plurality of claim 
2, wherein the means for transferring the reference frames 
using the plurality of reference frame translation tools 
further comprises: 15 

a first transfer sub-module for transferring a set of 
representation/non -homogeneous structure 
coordinates, and 

a second transfer sub-module for transferring a set of nQ 
non-homogeneous structure/representation coordi- 
nates. 

4. The interactive system according to claim 3, wherein 
the first transfer sub-module comprises: 

means for acquiring a set of coordinates (XM, YM, ZM), 05 
expressed in the first reference frame, of a point of the 
representation of the non-homogeneous structure to be 
transferred, by selection on the representation; 

means for calculating a set of corresponding coordinates 
(XP, YP, ZP), expressed in the second reference frame, 30 
on the non-homogeneous structure through a transfor- 
mation: 

{YP,YP, ZP}=M*N- 2 *{XM,YM,ZM} where M * N" 1 
represents a product of the matrix (M) and an inverse 
of the matrix (N), and 35 

means for processing, with the aid of the corresponding 
coordinates (YP, YP, ZP), to display a corresponding 
point on a surface of the non-homogeneous structure 
and to secure the intervention. 

5. The interactive system according to claim 3, wherein 40 
the second transfer sub-module comprises: 

means for acquiring a set of coordinates (XP, YP, ZP), 
expressed in the second reference frame, of a point of 
the non-homogeneous structure to be transferred; 45 

means for calculating a set of corresponding coordinates 
(XM YM, ZM), expressed in the first reference frame, 
of the representation through a transformation: 

{YM, YM, ZM}=N*M- n *{XP,ZP,ZP} where N*M~ 3 
represents the product of the matrix (N) and an inverse 50 
of the matrix (M); and, 

means for displaying the representation using the set of 
corresponding coordinates (YM, YM, ZM). 

6. The interactive system according to claim 1, wherein 
the means for generating the plurality of reference frame 55 
translation tools also generate, in association with the ref- 
erence frame translation tools, tools for taking into account 
a residual uncertainty which is based on the set of deviations 
between the set of coordinates of the plurality of images of 
the plurality of base points in the first reference frame and 60 
the set of coordinates of the base points, the tools for taking 
into account the residual uncertainty usable for displaying a 
set of contours in the representation whilst taking into 
account the residual uncertainties. 

7. The interactive system according to claim 1, wherein 65 
the means of dynamic displaying the three-dimensional 
image comprises: 
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a file containing digitized data from a set of two- 
dimensional images constituted by successive non- 
invasive tomographic sections of the non- 
homogeneous structure; 
means for calculating and reconstructing the three- 
dimensional image from the set of two-dimensional 
images; and 
a high-resolution display screen. 

8. The interactive system according to claim 7, wherein 
the means for calculating and reconstructing the three- 
dimensional image from the set of two-dimensional images 
comprises a program consisting of computer-aided design 
type software. 

9. The interactive system according to claim 1, wherein 
the means for determining the set of coordinates of the 
plurality of base points in the second reference frame 
comprises a three-dimensional probe equipped with a tactile 
tip for delivering a set of coordinates of the tactile tip in the 
said second reference frame. 

10. The interactive system according to claim 1, wherein 
the means for determining the set of coordinates of the 
plurality of base points is the second reference frame com- 
prises at least one of a set of optical sensors and a set of 
electromagnetic sensors. 

11. The interactive system according to claim 1, wherein 
a portion of the set of the plurality of base points of the 
reference structure comprises a plurality of marks positioned 
on a lateral surface of the non-homogeneous structure. 

12. The interactive system according to claim 11, wherein 
the plurality of marks are four in number and are distributed 
over the lateral surface so as to define a substantially 
symmetrical tetrahedron. 

13. The interactive system according to claim 1, wherein 
the means of intervention comprises: 

a guide arm to secure intervention in the region of the 
non-homogeneous structure, the guide arm having a 
position marked with respect to the second reference 
frame; and, 

an active intervention member whose position is marked 
with respect to the second reference frame. 

14. The interactive system according to claim 13, wherein 
the active intervention member is removable and selected 
from the group consisting of: 

tools for trephining; 
needles and implants; 

laser and radioisotope emission heads; and, sighting and 
viewing systems. 

15. The interactive system according to claim 1, wherein 
the means for transferring the plurality of reference frames 
establishes a coupling between a direction of visualization of 
the representation of the non-homogeneous structure on the 
display means and a direction of observation of the non- 
homogeneous structure and of the reference structure by the 
active intervention member. 

16. The interactive system according to claim 15, further 
comprising: 

a first module for visualizing a representation in a direc- 
tion given by two points; 
a second module for visualizing a representation in a 
direction given by an angle of elevation and an angle of 
azimuth. 
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Listing Of Claims 

1. An interactive system for local intervention inside a region of a non- 
homogeneous structure to which is connected a reference structure containing a 
plurality of base points, the interactive system comprising: 

means for dynamically displaying a three-dimensional image of a 
representation of the non-homogeneous structure and of the reference structure 
connected to the non-homogeneous structure, wherein the three-dimensional image 
also includes a plurality of images of the plurality of base points; 

means for determining a set of coordinates of the plurality of images of the 
plurality of base points in a first reference frame; 

means for fixing a position of the non-homogeneous structure and of the 
reference structure with respect to a second reference frame; 

means for determining a set of coordinates of the plurality of base points 
in the second reference frame; 

means of intervention comprising an active member whose position is 
determined with respect to the second reference frame; 

means for generating a plurality of reference frame translation tools for 
translating a plurality of reference frames from the first reference frame to the second 
reference frame and vice versa, based on the set of coordinates of the plurality of 
images of the plurality of base points in the first reference frame and of the set of 
coordinates of the plurality of base points in the second reference frame, in such a way 
as to reduce to a minimum at least one of a set of deviations between the set of 
coordinates of the plurality of images of the plurality of base points in the first reference 



frame and the set of coordinates of the base points, expressed in the first reference 
frame using the plurality of reference frame translation tools; 

means for defining, with respect to the first reference frame, a simulated 
origin of intervention and a simulated direction of intervention; and, 

means for transferring the plurality of reference frames using the plurality 
of reference frame translation tools to establish a bidirectional coupling between the 
simulated origin of intervention and the simulated direction of intervention and the 
position of the active member. 

2. The interactive system according to claim 1, wherein the plurality of 
reference frame translation tools comprise: 

means for creating a matrix (M) for transferring between the first reference 
frame and a first intermediate reference frame based on a set of coordinates of a set of 
three images of a set of three base points of the reference structure; 

means for creating a matrix (N) for transferring between the second 
reference frame and a second intermediate reference frame based on the set of 
coordinates of the set of three images of the set of three base points of the reference 
structure; and, 

means for validating matrix (M) and matrix (N) based on the set of three 
base points and the set of three images, such that at least one deviation between an 
expression for at least one additional base point in the second intermediate reference 
frame and an expression for at least one image point of the additional base point in the 
first intermediate reference frame is reduced to a minimum. 
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3. The interactive system according to plurality of claim 2, wherein the 
means for transferring the reference frames using the plurality of reference frame 
translation tools further comprises: 

a first transfer sub-module for transferring a set of representation/non- 
homogeneous structure coordinates, and 

a second transfer sub-module for transferring a set of non-homogeneous 
structure/representation coordinates. 

4. The interactive system according to claim 3, wherein the first transfer 
sub-module comprises: 

means for acquiring a set of coordinates (XM, YM, ZM), expressed in the 
first reference frame, of a point of the representation of the non-homogeneous structure 
to be transferred, by selection on the representation; 

means for calculating a set of corresponding coordinates (XP, YP, ZP), 
expressed in the second reference frame, on the non-homogeneous structure through a 
transformation: 

{YP.YP, ZP}=M*N.sup.-1 *{XM,YM,ZM} where M * N.sup.-1 represents a 
product of the matrix (M) and an inverse of the matrix (N), and 

means for processing, with the aid of the corresponding coordinates (YP, 
YP, ZP), to display a corresponding point on a surface of the non-homogeneous 
structure and to secure the intervention. 
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5. The interactive system according to claim 3, wherein the second transfer 
sub-module comprises: 

means for acquiring a set of coordinates (XP, YP, ZP), expressed in the 
second reference frame, of a point of the non-homogeneous structure to be transferred; 

means for calculating a set of corresponding coordinates (XM YM, ZM), 
expressed in the first reference frame, of the representation through a transformation: 

{YM, YM, ZM}=N*M.sup.-1 *{XP,ZP,ZP} where N*M.sup.-1 represents the 
product of the matrix (N) and an inverse of the matrix (M); and, 

means for displaying the representation using the set of corresponding 
coordinates (YM, YM, ZM). 

6. The interactive system according to claim 1, wherein the means for 
generating the plurality of reference frame translation tools also generate, in association 
with the reference frame translation tools, tools for taking into account a residual 
uncertainty which is based on the set of deviations between the set of coordinates of the 
plurality of images of the plurality of base points in the first reference frame and the set 
of coordinates of the base points, the tools for taking into account the residual 
uncertainty usable for displaying a set of contours in the representation whilst taking into 
account the residual uncertainties. 

7. The interactive system according to claim 1, wherein the means of 
dynamic displaying the three-dimensional image comprises: 
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a file containing digitized data from a set of two-dimensional images 
constituted by successive non-invasive tomographic sections of the non-homogeneous 
structure; 

means for calculating and reconstructing the three-dimensional image 
from the set of two-dimensional images; and 
a high-resolution display screen. 

8. The interactive system according to claim 7, wherein the means for 
calculating and reconstructing the three-dimensional image from the set of two- 
dimensional images comprises a program consisting of computer-aided design type 
software. 

9. The interactive system according to claim 1, wherein the means for 
determining the set of coordinates of the plurality of base points in the second reference 
frame comprises a three-dimensional probe equipped with a tactile tip for delivering a 
set of coordinates of the tactile tip in the said second reference frame. 

10. The interactive system according to claim 1, wherein the means for 
determining the set of coordinates of the plurality of base points is the second reference 
frame comprises at least one of a set of optical sensors and a set of electromagnetic 
sensors. 
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1 1 . The interactive system according to claim 1 , wherein a portion of the set of 
the plurality of base points of the reference structure comprises a plurality of marks 
positioned on a lateral surface of the non-homogeneous structure. 

12. The interactive system according to claim 11, wherein the plurality of 
marks are four in number and are distributed over the lateral surface so as to define a 
substantially symmetrical tetrahedron. 

13. The interactive system according to claim 1, wherein the means of 
intervention comprises: 

a guide arm to secure intervention in the region of the non-homogeneous 
structure, the guide arm having a position marked with respect to the second reference 
frame; and, 

an active intervention member whose position is marked with respect to 
the second reference frame. 

14. The interactive system according to claim 13, wherein the active 
intervention member is removable and selected from the group consisting of: 

tools for trephining; 
needles and implants; 

laser and radioisotope emission heads; and, sighting and viewing 

systems. 
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15. The interactive system according to claim 1, wherein the means for 
transferring the plurality of reference frames establishes a coupling between a direction 
of visualization of the representation of the non-homogeneous structure on the display 
means and a direction of observation of the non-homogeneous structure and of the 
reference structure by the active intervention member. 

16. The interactive system according to claim 15, further comprising: 

a first module for visualizing a representation in a direction given by two 

points; 

a second module for visualizing a representation in a direction given by an 
angle of elevation and an angle of azimuth. 

17. (Canceled) 

18. (Canceled) 
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19, (Twice Amended) An interactive system for intervention inside a region 
of a patient, said interactive system comprising: 

a device operable to receive image data of the region of the patient 

wherein the image data includes image data of a first reference structure to establish an 
image reference frame for the region of the patient; 

a second reference structure positioned relative to the patient to establish 

a patient reference frame for the region of the patient; 

a controller operable to correlate the position of the first reference 

structure in the image reference frame with the position of the second reference 
structure in the patient reference frame; 

an active member operable to perform the intervention; and 

a tracking system operable to determine a position of at least the second 

reference structure and a position of the active member and configured to transmit the 
determined positions of the second reference structure and the active member to the 
controller; 

wherein the controller is configured to determine the position of the active 

member based on the determined position of at least the active member and the 
correlation of the first reference structure and the second reference structure. 

20. (previously presented) The interactive system as defined in Claim 19 

wherein the first reference structure includes a plurality of base points. 
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21. (previously presented) The interactive system as defined in Claim 20 

wherein the second reference structure includes a plurality of tracking markers. 

22. (previously presented) The interactive system as defined in Claim 19 

wherein the second reference structure includes a plurality of tracking markers. 

23. (previously presented) The interactive system as defined in Claim 22 

wherein the plurality of tracking markers are attached to the patient. 

24. (previously presented) The interactive system as defined in Claim 19 

wherein the second reference structure is attached to the patient. 

25. (previously presented) The interactive system as defined in Claim 19 
wherein the first reference structure is attached to the patient. 

26. (previously presented) The interactive system as defined in Claim 21 

wherein the plurality of base points are generated from the plurality of tracking markers. 

27. (previously presented) The interactive system as defined in Claim 20 

wherein the plurality of base points are at least one of a plurality of notable points on the 
patient and marks fixed to the patient 
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28. (previously presented) The interactive system as defined in Claim 27 

wherein the notable points are selected from a group comprising a head, eyebrows, 
temples, frontal medial point, an apex of a skull, a center of gravity of an orbits of the 
eves and a combination thereof. 

29, (previously presented) The interactive system as defined in Claim 19 

wherein the tracking system includes a marker device operable to determine a position 
of the second reference structure in relation to the patient reference frame. 

30. (Amended) The interactive system as defined in Claim 29 wherein the 

marker device includes a telemetry system operable to determine the position of the 
second reference structure in the patient reference frame and transmit the determined 
position to the controller, wherein the controller is operable to perform the correlation at 
least with the transmitted determined position. 

31. (previously presented) The interactive system as defined in Claim 30 

wherein the telemetry system is an electromagnetic telemetry system. 

32. (previously presented) The interactive system as defined in Claim 31 
wherein the second reference structure includes electromagnetic tracking markers, 
wherein the electromagnetic telemetry system is operable to determine the position of 
the electromagnetic tracking markers of the second reference structure in relation to the 
patient reference frame. 
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33. (previously presented) The interactive system as defined in Claim 32, 

wherein the electromagnetic tracking markers are transmitters and the electromagnetic 
telemetry system is an electromagnetic sensor. 

34. (previously presented) The interactive system as defined in Claim 30 

wherein the telemetry system is an optical telemetry system. 

35. (Amended) The interactive system as defined in Claim 34 wherein the 

optical telemetry system includes at least one of a video camera or an infrared camera 
to image at least the second reference structure and configured to plot points of the 
second reference structure. 

36. (previously presented) The interactive system as defined in Claim 34 
wherein the second reference structure includes optical tracking markers, wherein the 
optical telemetry system is operable to determine the position of the optical tracking 
markers of the second reference structure in relation to the patient reference frame. 

37. (previously presented) The interactive system as defined in Claim 34 

wherein the optical telemetry system utilizes position and shape recognition to identify 
the second reference structure. 

38. (previously presented) The interactive system as defined in Claim 29 

wherein the marker device includes a three-dimensional probe. 
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39. (previously presented) The interactive system as defined in Claim 38 
wherein the three-dimensional probe includes a tactile tip operable to engage the 
second reference structure. 

40. (previously presented) The interactive system as defined in Claim 38 

wherein the three-dimensional probe is roboticallv manipulated, such that the 
instantaneous position of the three-dimensional probe is known. 

41. (previously presented) The interactive system as defined in Claim 29 

wherein the marker device includes a set of cameras operable to determine the position 
of the second reference structure in relation to the patient reference frame. 

42. (previously presented) The interactive system as defined in Claim 41 

wherein the set of cameras are selected from video and infrared cameras. 

43. (previously presented) The interactive system as defined in Claim 29 

wherein the marker device is a laser beam emission system operable to illuminate the 
second reference structure to determine a position of the second reference structure in 
relation to the patient reference frame. 

44. (previously presented) The interactive system as defined in Claim 20 

wherein the controller further includes a graphical tool operable to identify the plurality of 
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base points of the first reference structure in the image data of the image data reference 
frame. 

45. (previously presented) The interactive system as defined in Claim 44 

wherein the graphical tool is a mouse in communication with the controller. 

46. (previously presented) The interactive system as defined in Claim 19 

wherein the first reference structure is generated from the second reference structure. 

47, (canceled) 

48. (Amended) The interactive system as defined in Claim 19 wherein the 

active member is selected from a group comprising a trephining tool, a needle, a laser, 
a radioscope emission head, an endoscopic viewing system, a tool used in the 
intervention, an implant, a sighting system, a microscope, and combinations thereof. 

49. (Amended) The interactive system as defined in Claim 19 further 

comprising a telemetry system operable to determine the position of the active member 
in the patient reference frame, said telemetry system in communication with the 
controller. 
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50. (previously presented) The interactive system as defined in Claim 49 

wherein the position information of the active member is six degree of freedom 
information in relation to the patient reference frame. 

51. (Amended) The interactive system as defined in Claim W47]] 19 wherein 

the device includes a display operable to display the image data of the region of the 
patient in relation to the image reference frame. 

52. (previously presented) The interactive system as defined in Claim 51 

wherein the controller is further operable to determine a reference origin of intervention 
and a direction of intervention and said display is further operable to display the 
reference origin of intervention and direction of intervention. 

53. (previously presented) The interactive system as defined in Claim 51 
wherein the controller is further operable to model a reference origin of intervention and 
a direction of intervention and said display is further operable to display the modeled 
reference origin of intervention and direction of intervention. 

54. (Amended) The interactive system as defined in Claim 51 wherein the 

display is further operable to display the real-time position of the active member in the 
image reference frame based on the determined position of the active member with the 
tracking system. 
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55. (previously presented) The interactive system as defined in Claim 51 

wherein the display is further operable to display image data relative to a direction of 
intervention of the active member. 

56. (previously presented) The interactive system as defined in Claim 55 

wherein the image data is displayed perpendicular to a direction of intervention of the 
active member. 

57, (previously presented) The interactive system as defined in Claim 51 

wherein the controller is further operable to simulate an optimal trajectory of advance of 
the active member and said display is operable to display the optimal trajectory in the 
image data relative to the image reference frame. 

58. (previously presented) The interactive system as defined in Claim 57 
wherein movement of the active member is steered to the optimal trajectory to carry out 
a programmed intervention. 

59. (Amended) The interactive system as defined in Claim 19 wherein the 

active member is roboticallv controlled. 

60. (previously presented) The interactive system as defined in Claim 19 

wherein the image data is at least one of a magnetic resonance image data, a 
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tomographic image data, a radiographic image data, x-ray image data, and 
combinations thereof. 

61 (previously presented) The interactive system as defined in Claim 19 

wherein the device is operable to construct three-dimensional images from captured 
two-dimensional images. 
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62. (previously presented) An interactive system for intervention inside a region 
of a patient, said interactive system comprising: 

a device operable to receive image data of the region of the patient 

wherein the image data includes image data of a first reference structure to establish an 
image reference frame for the region of the patient; 

a second reference structure positioned relative to the patient to establish 

a patient reference frame for the region of the patient; and 

a controller operable to correlate the position of the first reference 

structure in the image reference frame with the position of the second reference 
structure in the patient reference frame; 

wherein the device is operable to construct three-dimensional images from 

captured two-dimensional images; 

wherein the controller is operable to superimpose two-dimensional image 

data on the three-dimensional images wherein any change in soft external parts of the 
patient can be visualized as compared with the image captured by the imaging device. 
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63. (previously presented) An interactive system for intervention inside a 

region of a patient, said interactive system comprising: 

a device operable to receive image data of the region of the patient, 

wherein the image data includes image data of a first reference structure to establish an 
image reference frame for the region of the patient; 

a second reference structure positioned relative to the patient to establish 

a patient reference frame for the region of the patient; 

a controller operable to correlate the position of the first reference 

structure in the image reference frame with the position of the second reference 
structure in the patient reference frame; and 

an active member operable to perform the intervention; 

wherein the device includes a display operable to display the image data 

of the region of the patient in relation to the image reference frame; 

wherein the controller is further operable to determine residual uncertainty 

which is used to represent a contour with dimensions larger than those which would 
normally be represented and the display is operable to display the residual uncertainty 
of the contour. 

64. (previously presented) The interactive system as defined in Claim 63 

wherein the contour is a display of an active member and a representation of residual 
uncertainty in order to reduce the chance of traversing undesired structures. 
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65. (previously presented) The interactive system as defined in Claim 19 
wherein the controller is further operable to correlate map data in a map reference 
frame with the patient reference frame. 

66. (Amended) The interactive system as defined in Claim 19 wherein the 

intervention is at least one of a neurosurgery, orthopedic surgery, cranial surgery, and 
combinations thereof. 

67. (previously presented) The interactive system as defined in Claim 19 

wherein the second reference structure is fixed to a head set. 

68. (previously presented) The interactive system as defined in Claim 60 

wherein the head set is further fixed to an operating table. 

69. (previously presented) The interactive system as defined in Claim 19 

wherein the device further includes memory operable to store the image data. 

70, (previously presented) The interactive system as defined in Claim 19 

wherein the device is a first computer. 

71. (previously presented) The interactive system as defined in Claim 70 

wherein the controller is a second computer. 
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72. (previously presented) The interactive system as defined in Claim 71 

wherein the first computer and the second computer is a single work station. 
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73. (Twice Amended) An interactive system for intervention inside a region 
of a patient, said interactive system comprising: 

a device operable to receive image data of the region of the patient, 

wherein the image data includes image data of a first reference structure to establish an 
image reference frame for the region of the patient; 

a second reference structure positioned relative to the patient to establish 

a patient reference frame for the region of the patient; 

a controller operable to correlate the position of the first reference 

structure in the image reference frame with the position of the second reference 
structure in the patient reference frame; 

an active member operable to perform the intervention inside the region of 

the patient; 

a tracking system operable to track the position of the active member in 

relation to the patient reference frame, the tracking system being in communication with 
the controller to transmit the tracked position of the active member as position 
information to the controller wherein the controller is operable to determine the position 
of the active member relative to the image reference frame; and 

a display operable to display the real-time position of the active member in 

the image reference frame based on the controller determined position of the active 
member based on the tracked position of the active member from the tracking system, 
wherein the controller is configured to generate a representation of the active member 
that is displayed on the display relative to a display of the received image data, 
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74. (previously presented) The interactive system as defined in Claim 73 
wherein the active member is selected from a group comprising a trephining tool, a 
needle, a laser, a radioscope emission head, an endoscopic viewing system, a tool 
used in the intervention, an implant a sighting system, a microscope, and combinations 
thereof. 

75. (previously presented) The interactive system as defined in Claim 73 

wherein the position information of the active member is six degree of freedom 
information in relation to the patient reference frame. 

76. (previously presented) The interactive system as defined in Claim 

73 wherein the tracking system that tracks the position of the active member is a 
telemetry system in communication with the controller. 

77. (previously presented) The interactive system as defined in Claim 73 

wherein the active member is robotically controlled. 

78. (previously presented) The interactive system as defined in Claim 73 

wherein the image data is at least one of a magnetic resonance image data, a 
tomographic image data, a radiographic image data, x-rav image data, and 
combinations thereof. 
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79. (previously presented) The interactive system as defined in Claim 73 

wherein the controller is further operable to determine a reference origin of intervention 
and a direction of intervention and said display is further operable to display the 
reference origin of intervention and direction of intervention, 

80. (previously presented) The interactive system as defined in Claim 73 
wherein the first reference structure includes a plurality of base points. 

81. (previously presented) The interactive system as defined in Claim 80 

wherein the second reference structure includes a plurality of tracking markers. 

82. (previously presented) The interactive system as defined in Claim 81 

wherein the plurality of base points are generated by the plurality of tracking markers. 

83. (previously presented) The interactive system as defined in Claim 73 

wherein the second reference structure is attached to the patient. 

84. (previously presented) The interactive system as defined in Claim 73 

wherein intervention is at least one of a neurosurgery, orthopedic surgery, cranial 
surgery intervention, and combinations thereof. 

85. (previously presented) The interactive system as defined in Claim 73 

wherein the second reference structure is fixed to a head set. 
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86. (Amended) The interactive system as defined in Claim 73 wherein the 

display forms part of the device and wherein the image data received is acquired image 
data of the region of the patient and is displayed on the display, further wherein the 
representation of the active member is displayed on the acquired image data of the 
region of the patient. 
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87. (Twice Amended) A method for performing an image guided 

intervention inside a region of a patient, said method comprising: 

accessing a first image data of the region of the patient captured with an 

imaging system where the first image data includes image data of a first reference 
structure; 

identifying the first reference structure in the first image data to establish 

an image reference frame; 

identifying a second reference structure relative to the patient to establish 

a patient reference frame; 

correlating the position of the first reference structure in the image 

reference frame in the first image data with the position of the second reference 
structure in the patient reference frame; and 

tracking an active member at least to determine a position of the active 

member in the patient reference frame to determine a location of the active member 
based on the tracking of the active member and transmitting the determined position in 
the patient refrence frame for display on a display device relative to the image reference 
frame of the first image data based at least on the correlation of the first reference 
structure and the second reference structure. 

88. (previously presented) The method as defined in Claim 87 further 

comprising attaching a plurality of tracking markers to the patient where the tracking 
markers form the second reference structure. 
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89. (previously presented) The method as defined in Claim 88 further 

comprising identifying the position of the tracking markers in the patient reference frame 
using a telemetry system. 

90. (Amended) The method as defined in Claim 89 further comprising 

transmitting from the tracking markers a signal and receiving the transmitted signal with 
an electromagnetic sensor to identify the position of the second reference structure in 
the patient reference frame. 

91. (previously presented) The method as defined in Claim 87 wherein 

identifying the first reference structure includes identifying a plurality of base points 
visible in the image data. 

92. (previously presented) The method as defined in Claim 91 wherein 

identifying the plurality of base points includes identifying at least one of notable points 
on the patient as marks fixed to the patient representing the plurality of base points. 

93. (previously presented) The method as defined in Claim 92 wherein the 

notable points are selected from a group comprising a head, eyebrows, temporal point 
frontal medial point an apex of a skull, a center of gravity of an orbits of the eves and a 
combination thereof. 
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94. (previously presented) The method as defined in Claim 91 wherein the 

plurality of base points visible in the image data are generated from the plurality of 
tracking markers attached to the patient. 

95. (previously presented) The method as defined in Claim 87 further 

comprising attaching the second reference structure to the patient. 

96. (previously presented) The method as defined in Claim 87 further 

comprising displaying the image data of the region of the patient, including displaying 
the first reference structure. 

97. (previously presented) The method as defined in Claim 87 further 

comprising performing an intervention on the patient with an active member. 

98. (Canceled) 

99. (Amended) The method as defined in Claim 96 further comprising: 

displaying the position of the active member as a representation of the 

active member in the accessed first image data that is captured image data that is 
correlated to the patient based on the correlation and displayed on a display device with 
the position of the active member being correlated between the patient reference frame 
defined by the first reference structure fixed to the patient and the image reference 
frame based on the tracking of the active member. 
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100. (Amended) The method as defined in Claim 99 further comprising 
identifying the position of the active member with a telemetry system by transmitting the 
tracked location of the active member for displaying the representation of the active 
member. 

101. (previously presented) The method as defined in Claim 99 further 

comprising displaying a reference origin of intervention and a direction of intervention in 
the image data. 

102. (previously presented) The method as defined in Claim 101 further 

comprising tracking the position of the active member relative to the reference origin of 
intervention and the direction of intervention. 

103. (previously presented) The method as defined in Claim 87 further 

comprising converting two-dimensional image data to three-dimensional image data. 

104. (previously presented) The method as defined in Claim 97 wherein the 

intervention is selected from at least one of a neurosurgery, orthopedic surgery, cranial 
surgery, and combinations thereof. 

105. (previously presented) The method as defined in Claim 95 further 

comprising attaching the second reference structure to a head set. 
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ATTACHMENT J 



Taylor, Michael 



From: Taylor, Michael 

Sent: Wednesday, November 09, 2011 3:30 PM 

To: 'nchdou^fo-rothschild.fr 1 ; 'jbthiebaut@fo-rothschild.fr'; 'Joel. henrion@wanadoo.fr'; 

Uf.uhl@free.fr'; 'Michel Scriban'; 'jf.uhl@wanadoo.fr'; jefuh^gmaiLcom'; 

, m.scriban@nelixa.com' 
Cc: Warner, Rick; Hall, Stephanie; Neal, Patrick 

Subject: RE: Response Required November 14, 2011 - Re-issue of U.S. Patent No. 5,868,675 

(5074a-000013/REA) [HDP-TroyJLegal.FID2452556] 
Attachments: DHLJracking„sentJ\Iov-l-2011 (all).PDF 

Dear Michel Scriban, Joel Henrion, Jean Francois UHL, and Jean-Baptiste Thiebaut: 

Attached is a tracking list from DHL indicating that the packages we sent were delivered and signed for on November 3, 
2011. 

This is a reminder that we requested the executed Supplemental Reissue Declaration by November 14, 2011 to respond 
in good time to the U.S. Patent Office. 

Best Regards, 

Michael Taylor 



I t J!Su t"" t t\.i T""" *^! T MiMil Michael L.Taylor [ Patent Attorney 

r^^™^&.a= ram o | 248.641.1600 F j 248.641.0270 D j 248.641.1289 
D ICKEY IP Causes Worldwide 



From: Taylor, Michael 

Sent: Tuesday, November 01, 2011 4:10 PM 

To: 'jbthiebaut@fo-rothschild.fr'; 'JoeLhenrion@wanadoo.fr 1 ; 'jf.uhl@free.fr'; 'Michel Scriban 1 ; 'jf.uhl@wanadoo.fr 1 
Cc: Warner, Rick; Hall, Stephanie; Neal, Patrick 

Subject: Response Required November 14, 2011 - Re-issue of U.S. Patent No. 5,868,675 (5074a-000013/REA) [HDP- 

TroyJ_egal.FID2452556] 

Importance: High 

Michel Scriban 
72 Chemin de Crapon 
69360 Ternay, France 
m.scriban(g>nelixa.fr 

Joel Henrion 

17; Route de Chalone 

51600 Suippes, France 

Joel.henrion@wanadoo.fr 

Jean Francois UHL 
199 avenue du Maine 
Paris, France 75014 



l 



Jf.uhl@free.fr and Jf.uhl@wanadoo.fr 

Jean-Baptiste Thiebaut 
42 boulevard Saint-Marcel 
Paris, France 75005 
ibthiebaut@fo-rothschild.fr 



Dear Sirs, 

The attached documents are for your review, consideration, execution, and return to us. We are also sending paper 
copies to the addresses noted above. 

We thank you for your consideration and look forward to you swift response. 

Sincerely, 

Michael L Taylor 



Michael L. Taylor off,ce: ^'^i.ieoo 

Patent Attorney Fax: 248.641.0270 

Y Direct: 248.641.1289 



I MlyAt PirsJE-SS IBB $445 Corporate Dr, Suite 200 

| mmmsmm pitTKEY Troy/ MI 48098 

I IP Causes Worldwide Bio ♦ vCard 

I = 

Metropolitan \ Detroit ♦ Portland ♦ St. Louis ♦ Washington, DC 
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English Contact Center Tools Country Profile 
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Track DHL Express Shipments 

DHL's online tracking is the fastest way to find out where your shipment is. No need to call 
Customer Service when we can offer you real-time details of your shipment's progress as it 
speeds through the DHL Network on the way to its destination. 
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Result Summary Show AH 



Waybill: 8809998164 Thursday, November 03, 2011 at 10:06 

f-yi Origin Service Area: > ROMULUS, Ml - TROY - USA 

| Y\ Signed for by: TEIXEIRA 

" J Destination Service Area: > ORLY - PARIS - 

FRANCE 



Thursday, November 03, 2011 Location Time 

1 1 Delivered - Signed for by : TEIXEIRA ORLY - FRANCE 1 0:06 

10 With delivery courier ORLY - FRANCE 08:48 

9 Arrived at Sort Facility ORLY - FRANCE ORLY - FRANCE 02:41 

8 Departed Facility in PARIS - FRANCE PARIS - FRANCE 00:50 

Wednesday, November 02, 201 1 Location Time 

7 Processed at PARIS - FRANCE PARIS - FRANCE 20:22 

6 Departed Facility in CINCINNATI HUB - CINCINNATI HUB, OH - USA 05:23 
USA 

5 Processed at CINCINNATI HUB - USA CINCINNATI HUB, OH - USA 04:24 

4 Arrived at Sort Facility CINCINNATI HUB - CINCINNATI HUB, OH - USA 02:37 
USA 

Tuesday, November 01 , 201 1 Location Time 

3 Departed Facility in ROMULUS - USA ROMULUS, Ml - USA 23:37 

2 Processed at ROMULUS - USA ROMULUS, Ml - USA 23:36 

1 Shipment picked up ROMULUS, Ml - USA 1 7:50 
Hide Details 



Waybill: 8809998153 Thursday, November 03, 2011 at 11:41 

r-ji Origin Service Area: > ROMULUS, Ml - TROY - USA 

| vnl Signed lor by: RNMJ 

h — A Destination Service Area: > ORLY - PARIS - 

FRANCE 



Thursday, November 03, 2011 Location Time 

1 1 Delivered - Signed for by : RNMJ ORLY - FRANCE 1 1 :41 

1 0 With delivery courier ORLY - FRANCE 07:31 

9 Arrived at Sort Facility ORLY - FRANCE ORLY - FRANCE 02:41 

8 Departed Facility in PARIS - FRANCE PARIS - FRANCE 00:50 

Wednesday, November 02, 201 1 Location Time 

7 Processed at PARIS - FRANCE PARIS - FRANCE 20:22 

6 Departed Facility in CINCINNATI HUB - CINCINNATI HUB, OH - USA 05:23 
USA 

5 Processed at CINCINNATI HUB - USA CINCINNATI HUB, OH - USA 04:24 

4 Arrived at Sort Facility CINCINNATI HUB - CINCINNATI HUB, OH - USA 02:37 
USA 

Tuesday, November 01 , 201 1 Location Time 

3 Departed Facility in ROMULUS - USA ROMULUS, Ml - USA 23:37 

2 Processed at ROMULUS - USA ROMULUS, Ml - USA 23:36 

1 Shipment picked up ROMULUS, Ml - USA 17:50 
Hide Details 



Waybill: 8809998142 Thursday, November 03, 201 1 at 14:04 

Origin Service Area: > ROMULUS, Ml - TROY - USA 

i/\ Signed for by: HENION 

— 5 Destination Service Area: > ORLY - SUIPPES - 
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Thursday, November 03, 201 1 


Location 


Time 


"1 ^ rV.H.»*r«rl. O^nojj f 0f j->y • HENION 

Deutsche Post DHl>ier 


ORLY - FRANCE 
ORLY - FRANCE 


Sitemlaj)' Accessibility Legal Using DHL Websites 
201 ? W$Hl International GmbH. All rights reserved. 


- Mftiveu ai uenvery Facility in ORLY - 
FRANCE 


ORLY - FRANCE 


09:31 


1 1 Departed Facility in ORLY - FRANCE 


ORLY - FRANCE 


07:58 


10 Processed at ORLY - FRANCE 


ORLY - FRANCE 


04:18 


9 Arrived at Sort Facility ORLY - FRANCE 


ORLY - FRANCE 


02:41 


8 Departed Facility in PARIS - FRANCE 


PARIS - FRANCE 


00:50 



Wednesday, November 02, 2011 

7 Processed at PARIS - FRANCE 

6 Departed Facility in CINCINNATI HUB - 
USA 

5 Processed at CINCINNATI HUB - USA 

4 Arrived at Sort Facility CINCINNATI HUB - 
USA 



Location Time 

PARIS - FRANCE 20:22 

CINCINNATI HUB, OH - USA 05:23 

CINCINNATI HUB, OH - USA 04:24 

CINCINNATI HUB, OH - USA 02:37 



Tuesday, November 01, 2011 

3 Departed Facility in ROMULUS - USA 
2 Processed at ROMULUS - USA 
1 Shipment picked up 
Hide Details 

| t/\ Signed for by: SCRIBAN 



Location Time 

ROMULUS, Ml - USA 23:37 

ROMULUS, Ml - USA 23:36 

ROMULUS, Ml - USA 17:50 

Thursday, November 03, 2011 at 14:23 

Origin Service Area: > ROMULUS, Mi - TROY - USA 

Destination Service Area: > LYON - TERNAY - 
FRANCE 



Thursday, November 03, 2011 



15 


Delivered - Signed for by : SCRIBAN 


14 


With delivery courier 


13 


Arrived at Delivery Facility in LYON - 




FRANCE 


12 


Departed Facility in LYON - FRANCE 


11 


Processed at LYON - FRANCE 


10 


Arrived at Sort Facility LYON - FRANCE 


9 


Departed Facility in LEIPZIG - GERMANY 


8 


Processed at LEIPZIG - GERMANY 


Wednesday, November 02, 2011 


7 


Arrived at Sort Facility LEIPZIG - 




GERMANY 


6 


Departed Facility in CINCINNATI HUB - 




USA 


5 


Processed at CINCINNATI HUB - USA 


4 


Arrived at Sort Facility CINCINNATI HUB - 




USA 


Tuesday, November 01, 2011 


3 


Departed Facility in ROMULUS - USA 


2 


Processed at ROMULUS - USA 


1 


Shipment picked up 


Hide Details 



Location Time 

LYON -FRANCE 14:23 

LYON - FRANCE 09:48 

LYON -FRANCE 08:11 

LYON - FRANCE 07:37 

LYON -FRANCE 06:18 

LYON - FRANCE 05:58 

LEIPZIG -GERMANY 03:37 

LEIPZIG -GERMANY 00:22 

Location Time 

LEIPZIG -GERMANY 23:08 

CINCINNATI HUB, OH - USA 08:48 

CINCINNATI HUB, OH - USA 05:09 

CINCINNATI HUB, OH - USA 02:37 

Location Time 

ROMULUS, Ml - USA 23:37 

ROMULUS, Ml - USA 23:36 

ROMULUS, Ml - USA 17:50 
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ATTACHMENT K 



Taylor, Michael 



From: Joel HENRION <joeI.henrion@wanadoo.fr> 

Sent: Thursday, November 10, 2011 1:58 AM 

To: Taylor, Michael; 'nchdou^fo-rothschildir' 

Cc: Warner, Rick; Hall, Stephanie; Neal, Patrick 

Subject: RE: Response Required November 14, 2011 - Re-issue of U.S. Patent No. 5,868,675 

(5074a-000013/REA) [HDP-TroyJ_egal.FID2452556] 

Attachments: DHLJracking_sent_Nov-l-2011_-aII-.PDF 



Je vous prie de prendre connaissance de ma remarque sur le document PDF joint a cet envoi... 
Cordialement, Joel HENRION 



> Message du 09/11/11 21:30 

> De : "Taylor, Michael" 

> A : n, nchdoul@fo-rothschild.fr ,M , "'jbthiebaut@fo-rothschildir ,n , "'JoeLhenrk^wanadoo.fr'" 

<'Joel. henrion@wanadoo.fr>, "'Jf.uhl@free.fr m <'Jf.uhl@free.fr>, "'Michel Scriban'" , "'jf.uh^wanadoo.fr'" , 
m jef.uhl@gmail.com ,H , " , m.scriban@nelixa.com M, 

> Copie a : "Warner, Rick" , "Hall, Stephanie" , "Neal, Patrick" 

> Objet : RE: Response Required November 14, 2011 - Re-issue of U.S. Patent No. 5,868,675 (5074a-000013/REA) [HDP- 
Troy_Legal.FID2452556] 

> 
> 

Dear Michel Scriban, Joel Henrion, Jean Francois UHL, and Jean-Baptiste Thiebaut: 

Attached is a tracking list from DHL indicating that the packages we sent were delivered and signed for on November 3, 
2011. 

This is a reminder that we requested the executed Supplemental Reissue Declaration by November 14, 2011 to respond 
in good time to the U.S. Patent Office. 
Best Regards, 
Michael Taylor 

Michael L Taylor | Patent Attorney 



h-t^VF^fSJESS > q | 248.641.1600 F | 248.641.0270 D | 248.641.1289 

□ ICKEY 

> IP Causes Worldwide 



From: Taylor, Michael 

> Sent: Tuesday, November 01, 2011 4:10 PM 

>To: 'jbthiebaut@fo-rothschild.fr'; 'Joel. henrion@wanadoo.fr'; 'Jf.uhl@free.fr'; 'Michel Scriban'; 'jf.uhi@wanadoo.fr' 

> Cc: Warner, Rick; Hall, Stephanie; Neal, Patrick 

> Subject: Response Required November 14, 2011 - Re-issue of U.S. Patent No. 5,868,675 (5074a-000013/REA) [HDP- 
Troy_Legal.FID2452556] 

> Importance: High 



Michel Scriban 
72 Chemin de Crapon 
69360 Ternay, France 
m.scriban@nelixa.fr 



i 



Joel Henrion 

17, Route de Chalone 

51600 Suippes, France 

Joel.henrion(®wanadoo.fr 

Jean Francois UHL 

199 avenue du Maine 

Paris, France 75014 

Jf.uhl(5>free.fr and Jf.uhl@wanadoo.fr 

Jean-Baptiste Thiebaut 
42 boulevard Saint-Marcel 
Paris, France 75005 
ibthiebaut@fo-rothschild.fr 



Dear Sirs, 

The attached documents are for your review, consideration, execution, and return to us. We are also sending paper 
copies to the addresses noted above. 

We thank you for your consideration and look forward to you swift response. 

Sincerely, 

Michael L Taylor 



Michael L Taylor 

> Patent Attorney 



Office: 248.641.1600 
>Fax: > 248.641.0270 

> Direct: > 248,641.1289 



; > 5445 Corporate Dr, Suite 200 

MAJRMESS «BM > Troy, Ml 48098 

^^m:m:m DICKEY 
! IP Causes Worldwide Bio ♦ vCard 

( " ■ ■■ m ini m ii m i n i m i i i. mm ii m iiiiiiiii ■■— ■■■■ 

.Metropolitan ( Detroit ♦ Portland ♦ St. Louis * Washington, DC 



HARNESS, DICKEY & PiERCE, PLC LEGAL NOTICE: The information contained in this transmission is intended only for the individuai(s) or entity(ies) to 
whom it is addressed, tt may contain information protected from use and/or disclosure by law, including information that is protected as confidential, attorney-client 
privileged, attorney work product and/or trade secrets. If the reader of this message is not the intended recipient, or an employee or agent responsible for 
delivering this message to the addressee, the reader Is hereby notified that any use. distribution or copying of this communication is strictly prohibited. If you 
believe you have received this facsimile or message in error, please immediately notify us at our expense by return mail or e-mail and permanently delete or 
destroy ail copies of the message. 

> To comply with U.S. Treasury regulations, we advise you that any discussion of Federal tax issues in this communication was not intended to be used, and 
cannot be used, by any person <i) to avoid penalties that may be imposed by the Interna! Revenue Service, or (ii) to promote, market or recommend to another 
party any matter addressed herein. 

> Any portions of this transmission containing controlled technical data are restricted by U.S. export laws and regulations, and may not be distributed or 
retransmitted to non-U.S. persons without appropriate licensing or a licensing exemption, Neither this information block, nor the signature block, nor the typed 
name of the sender, nor anything else in this message is intended to constitute an electronic signature unless a specific statement to the contrary is included. 
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ATTACHMENT L 



! 



| 



| 
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I 
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i 

i 



Taylor, Michael 



From: 
Sent: 
To: 
Cc: 

Subject: 
Attachments: 



Sophie PONCET <sophiexcgl@wanadoo.fr> 
Tuesday, November 15, 2011 9:28 AM 
Taylor, Michael 
GRANGE Maxime 
MEDTRONIC 

COURRIEL ME TAYLOR.pdf 



Importance: 



High 



Cher Monsieur, 



Mattre GRANGE, avocat au barreau de Lyon, repond a vos correspondances echangees notamment avec Michel SCRIBAN 
concernant le dossier en reference. 



Nous restons dans I'attente de votre reponse. 



Vous en remerciant par avance, 



Nous vous adressons nos sentiments les plus devoues. 



Maxime GRANGE 

ACCESS AVOCATS 

63 avenue de Saxe - 69003 LYON 

T 04 72 84 99 60 

F 04 72 84 99 69 



1 



AVOCATS 



A Lyon, le 15 novembre 2011 



ACCESS AVOCATS 

Solar! Hi 's \ yon en <:ours 
Palais T 1635 
Murivi MNAKF,£ 
Avocat associee 
M;mme GRANGE 
Avocat Eiiisode 



QiratopheNEYRET- 

AVOCATS 

Setoff 5o5 RCS l.yon 

Palais- T81S 



Membu;:- ti'nnc SCM 



Ombeline SIRAUDIN 
Palais -T 176 



Raphael del'RAT 

\\OCUl. 

Palais -T160S 



63 avenue de Saxe 
69003 LYON 

Tel :04 72 84 99 60 
Fax :04 72 34 99 69 



Monsieur Michael TAYLOR 
PAR MAIL : mItaylor@HDP,com 

ATTENTION : nouvelle toque 
Palais T 1635 

& : M SCRI8AN / HUL / THIEBAUT / HENRION - MEDTRONIC- 
MG/SP 



Mon Cher Confrere, 



J'interviens aupres de vous en ma qualite de conseil des inventeurs du 
brevet actuellement detenu par la societe MEDTRONIC, que vous 
representez. 

J'ai pris connaissance de vos echanges de correspondances recents. 

Pour le compte de MEDTRONIC, vous avez adresse a mes clients un 
dossier important, qui necessite plusieurs jours d'etude par chacun des 
experts devant intervenir dans ce cadre. 

Comme vous I' a indlque Michel SCRIBAN le 9 octobre dernier, ce process 
va engager des frais importants, qui ne seraient pas normal de laisser a la 
charge des personnes que vous sollicitez. 

En consequence de ce qui precede, je vous remercie de m'indiquer si 
votre cliente accepte de couvrir le montant global de ces frais, d'un 
montant net de 70.800 € hors frais de sequestre. 

Dans ce cadre, etant detenteur d'un compte special permettant le 
blocage de cette somme jusqu'a la finalisation de I'etude de votre dossier 
par mes clients, je vous propose d'intervenir a ce titre sur la base d'un 
montant forfaitaire de 4.200 €. 



Attention : au l er novembre 2011, les Societes <c Muriel LINARES - Avocats » et « AXIONS Avocats » 
fusionnent au profit d'une Selarl unique : ACCESS AVOCATS - RCS en cours 
Merci d'adresser les correspondances et reglements au nom de cette nouvelle societe. 



Le total representee done 75.000 €, sequestre inclus. 

Des que j'aurai regu votre confirmation de I'accord de la societe 
MEDTRONIC dans ce cadre, et que le virement de la somme nette de 
75.000 € aura ete constate sur le compte sequestre ouvert aupres de la 
CARPA a LYON, ville du Barreau ou je suis inscrits, I'etude des documents 
sera lancee sans aucun delai par mes clients et nous conviendront 
prealablement d'un delai de reponse a destination de votre cabinet. 

Compte tenu de ce qui precede, je reste dans I'attente de vous lire, 

Et vous prie de croire, Mon Cher Confrere, en mes sentiments les plus 
devoues. 

Maxime GRANGE 




Attention : au l ef novembre 2011, les Societes « Muriel UN ARES - Avocats » et « AXIONS Avocats » 
fusionnent au profit d'une Selarl unique : ACCESS AVOCATS - RCS en cours 
Merci d'adresser les correspondances et reglements au nom de cette nouvelle societe. 



ATTACHMENT M 



In Lyon, on November 15, 2011 



Mr. Michael TAYLOR 

By email : mltaylor@HDP.com 



To : nouvelle toque 
Palais T1635 



Dear colleague, 



I am addressing this letter to you in my capacity of Counsel of the inventors of the patent 
owned by the company MEDTRONIC, which you represent. 

I have read the communications from the previous correspondences. 

On behalf of MEDTRONIC, you have addressed to my clients an important file, which requires 
several days of study by each of the experts involved in this matter. 

As Michel SCRIBAN mentioned to you on October 9 of this year, this process will incur 
significant costs, which shouldn't be at the charge of the people of whom you are seeking the 
assistance. 

As a result of the foregoing, I would appreciate if you could let me know if your client agrees to 
cover the total amount of these expenses, a net amount of € 70,800 excluding costs of escrow. 

About that, as I am the holder of a special account on which this amount can be blocked until 
the completion of the study of your file by my clients, I propose to act on this matter on the 
basis of a lump sum of € 4,200. 

Such, the total of the expenses is € 75,000, including costs of escrow. 

As soon as I receive your confirmation that MEDTRONIC agrees to the above, and when the 
transfer of the net sum of € 75,000 has been made to the escrow account opened with the 
CARPA in LYON, city of the bar in which I am enrolled, my clients will start the study of the 
documents without delay and we will prior agree on the delay in which we should send the 
answer to your practice. 

Given the foregoing, I am looking forward to hearing from you. 

Sincerely, 
Maxime GRANGE 



! 



: 



ATTACHMENT N 



! 



Taylor, Michael 



From: 
Sent: 
To: 
Cc: 

Subject: 
Attachments: 



Taylor, Michael 

Wednesday, November 23, 2011 9:21 AM 
'Sophie PONCET' 

'GRANGE Maxime'; Warner, Rick; Neal, Patrick; Hall, Stephanie 

RE: MEDTRONIC (5074A-000013/REA) [HDP-TroyJ_egal.FID2452556] 

Current Claims.doc; Supplemental Reissue dedaration.pdf; US 5868675.pdf; 

Assignment_ChainJJSPTO.PDF; Assignment documnets (inventors to Diadex SA only) 

Medtronic, Incpdf; Itr to ALL inventors (English and French).PDF; Itr to atty Grange of 

inventors (dated Nov~22-2011.PDF 



Cher GRANGE, 



My email of yesterday appears to have been missing our letter. I apologize for this oversight. If you received our letter 
dated November 22, 2011 in yesterday's email, this email is a duplicate. 



From: Taylor, Michael 

Sent: Tuesday, November 22, 2011 11:57 AM 
To: Sophie PONCET 

Cc: GRANGE Maxime; Warner, Rick; Neal, Patrick; Hall, Stephanie 

Subject: RE: MEDTRONIC (5074A-000013/REA) [HDP»Troy_Legal.FID2452556] 

Cher GRANGE, 

We understand that you are an attorney for all of Michel Scriban, Joel Henrion, Jean Francois UHL, and Jean-Baptiste 
Thiebaut, 

Please see our attached letter dated November 22, 2011, which refers to our previous correspondence, also attached. 
Best Regards, 
Michael Taylor 



From: Sophie PONCET [mailto:sophie.ccgl@wanadoo.fr] 

Sent: Tuesday, November 15, 2011 9:28 AM 

To: Taylor, Michael 

Cc: GRANGE Maxime 

Subject: MEDTRONIC 

Importance: High 



Best Regards, 



Michael Taylor 





Michael t. Taylor j Patent Attorney 

O | 248.641.1600 F I 248.641.0270 D | 248.641.1289 
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Cher Monsieur, 



MaTtre GRANGE, avocat au barreau de Lyon, repond a vos correspondances echangees notamment avec 
Michel SCRIBAN concernant le dossier en reference. 

Nous restons dans I'attente de votre reponse. 

Vous en remerciant par avance, 

Nous vous adressons nos sentiments les plus devoues. 



Maxime GRANGE 

Access .Avocats 

63 avenue de Saxe - 69003 LYON 
T 04 72 84 99 60 
F 04 72 84 99 69 
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h/\rime:s5 mm 



Michael Taylor 

Direct Dial: 248-641-1289 

rnltaylor@hdp.com 



November 22, 201 1 



Acess Avocats EMAIL ONLY 

63 avenue de Saxe 
69003, LYON 
FRANCE 

Re: U.S. reissue application 09/784,829 which is a 
Reissue of the U.S. Patent 5,868,675 
Medtronic Ref. No, PC0000173.06 
HDP Ref. No. 5074A-000013/REA 



Dear Maxime Grange, 

We have reviewed your correspondence of November 15, 201 1 with our client, Medtronic, Inc. 

Based on your letter, we understand that you represent ALL of the inventors Michel Scriban, 
Joel Henrion, 

Jean Francois UHL, and Jean-Baptiste Thiebaut of the U.S. Pat. No. 5,868,675. 

We have attached hereto our correspondence to all ot the inventors sent on November 1, 2011 
in which we requested the executed Supplemental Re-issue Declarations by November 14, 
2011. As of this date we have NOT received any of the Supplemental Re-issue Declarations 
from any of the inventors. 

Please note that in our previous correspondence we state "Medtronic notes that no payment 
is required under the assignment agreement that was executed by all of the inventors 
on June 22, 1992 and deems your request for payment a refusal to execute and return 
the Supplemental Reissue Declaration." Medtronic, Inc. maintains that no further 
payment is required per the original assignment of the invention disclosed in U.S. Pat. No. 
5,868,675 from ALL of the inventors to DIADAX S.A. which, through proper assignment, now is 
owned by Medtronic, Inc. 



Harness, Dickey & Pierce, P.LC Attorneys and Counselors 
5445 Corporate Drive, Suite 200, Troy, Ml 48098 



Metropolitan: Detroit, Ml Si Louis, MO Washington, D.C Portland, OR 



Phone 248.641.1600 Fax 248,641.0270 



Iwwyv^hd 



Type Recipient Name Here 
November 21, 2011 
Page 2 of 2 



Thank you in advance for your clients' cooperation in this matter, however, if we do not receive 
executed Supplemental Reissue Declarations from your clients (the inventors) by November 28, 
2011 Medtronic, Inc. will proceed in the U.S. Patent Office with a petition to continue 
prosecution based orrtPfe inventors refusal to sign the declarations. 
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Listing Of Claims 

1. An interactive system for local intervention inside a region of a non- 
homogeneous structure to which is connected a reference structure containing a 
plurality of base points, the interactive system comprising: 

means for dynamically displaying a three-dimensional image of a 
representation of the non-homogeneous structure and of the reference structure 
connected to the non-homogeneous structure, wherein the three-dimensional image 
also includes a plurality of images of the plurality of base points; 

means for determining a set of coordinates of the plurality of images of the 
plurality of base points in a first reference frame; 

means for fixing a position of the non-homogeneous structure and of the 
reference structure with respect to a second reference frame; 

means for determining a set of coordinates of the plurality of base points 
in the second reference frame; 

means of intervention comprising an active member whose position is 
determined with respect to the second reference frame; 

means for generating a plurality of reference frame translation tools for 
translating a plurality of reference frames from the first reference, frame to the second 
reference frame and vice versa, based on the set of coordinates of the plurality of 
images of the plurality of base points in the first reference frame and of the set of 
coordinates of the plurality of base points in the second reference frame, in such a way 
as to reduce to a minimum at least one of a set of deviations between the set of 
coordinates of the plurality of images of the plurality of base points in the first reference 



frame and the set of coordinates of the base points, expressed in the first reference 
frame using the plurality of reference frame translation tools; 

means for defining, with respect to the first reference frame, a simulated 
origin of intervention and a simulated direction of intervention; and, 

means for transferring the plurality of reference frames using the plurality 
of reference frame translation tools to establish a bidirectional coupling between the 
simulated origin of intervention and the simulated direction of intervention and the 
position of the active member. 

2. The interactive system according to claim 1, wherein the plurality of 
reference frame translation tools comprise: 

means for creating a matrix (M) for transferring between the first reference 
frame and a first intermediate reference frame based on a set of coordinates of a set of 
three images of a set of three base points of the reference structure; 

means for creating a matrix (N) for transferring between the second 
reference frame and a second intermediate reference frame based on the set of 
coordinates of the set of three images of the set of three base points of the reference 
structure; and, 

means for validating matrix (M) and matrix (N) based on the set of three 
base points and the set of three images, such that at least one deviation between an 
expression for at least one additional base point in the second intermediate reference 
frame and an expression for at least one image point of the additional base point in the 
first intermediate reference frame is reduced to a minimum. 
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3. The interactive system according to plurality of claim 2, wherein the 
means for transferring the reference frames using the plurality of reference frame 
translation tools further comprises: 

a first transfer sub-module for transferring a set of representation/non- 
homogeneous structure coordinates, and 

a second transfer sub-module for transferring a set of non-homogeneous 
structure/representation coordinates. 

4. The interactive system according to claim 3, wherein the first transfer 
sub-module comprises: 

means for acquiring a set of coordinates (XM, YM, ZM), expressed in the 
first reference frame, of a point of the representation of the non-homogeneous structure 
to be transferred, by selection on the representation; 

means for calculating a set of corresponding coordinates (XP, YP, ZP), 
expressed in the second reference frame, on the non-homogeneous structure through a 
transformation: 

{YP,YP, ZP}=M*N.sup.-1 *{XM,YM,ZM} where M * N.sup.-1 represents a 
product of the matrix (M) and an inverse of the matrix (N), and 

means for processing, with the aid of the corresponding coordinates (YP, 
YP, ZP), to display a corresponding point on a surface of the non-homogeneous 
structure and to secure the intervention. 
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5. The interactive system according to claim 3, wherein the second transfer 
sub-module comprises: 

means for acquiring a set of coordinates (XP, YP, ZP), expressed in the 
second reference frame, of a point of the non-homogeneous structure to be transferred; 

means for calculating a set of corresponding coordinates (XM YM, ZM), 
expressed in the first reference frame, of the representation through a transformation: 

{YM, YM, ZM}=N*M.sup.-1 *{XP,ZP,ZP} where N*M.sup.-1 represents the 
product of the matrix (N) and an inverse of the matrix (M); and, 

means for displaying the representation using the set of corresponding 
coordinates (YM, YM, ZM). 

6. The interactive system according to claim 1, wherein the means for 
generating the plurality of reference frame translation tools also generate, in association 
with the reference frame translation tools, tools for taking into account a residual 
uncertainty which is based on the set of deviations between the set of coordinates of the 
plurality of images of the plurality of base points in the first reference frame and the set 
of coordinates of the base points, the tools for taking into account the residual 
uncertainty usable for displaying a set of contours in the representation whilst taking into 
account the residual uncertainties. 

7. The interactive system according to claim 1, wherein the means of 
dynamic displaying the three-dimensional image comprises: 
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a file containing digitized data from a set of two-dimensional images 
constituted by successive non-invasive tomographic sections of the non-homogeneous 
structure; 

means for calculating and reconstructing the three-dimensional image 
from the set of two-dimensional images; and 
a high-resolution display screen. 

8. The interactive system according to claim 7, wherein the means for 
calculating and reconstructing the three-dimensional image from the set of two- 
dimensional images comprises a program consisting of computer-aided design type 
software. 

9. The interactive system according to claim 1, wherein the means for 
determining the set of coordinates of the plurality of base points in the second reference 
frame comprises a three-dimensional probe equipped with a tactile tip for delivering a 
set of coordinates of the tactile tip in the said second reference frame. 

10. The interactive system according to claim 1, wherein the means for 
determining the set of coordinates of the plurality of base points is the second reference 
frame comprises at least one of a set of optical sensors and a set of electromagnetic 
sensors. 
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1 1 . The interactive system according to claim 1 , wherein a portion of the set of 
the plurality of base points of the reference structure comprises a plurality of marks 
positioned on a lateral surface of the non-homogeneous structure. 

12. The interactive system according to claim 11, wherein the plurality of 
marks are four in number and are distributed over the lateral surface so as to define a 
substantially symmetrical tetrahedron. 

13. The interactive system according to claim 1, wherein the means of 
intervention comprises: 

a guide arm to secure intervention in the region of the non-homogeneous 
structure, the guide arm having a position marked with respect to the second reference 
frame; and, 

an active intervention member whose position is marked with respect to 
the second reference frame. 

14. The interactive system according to claim 13, wherein the active 
intervention member is removable and selected from the group consisting of: 

tools for trephining; 
needles and implants; 

laser and radioisotope emission heads; and, sighting and viewing 

systems. 
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15. The interactive system according to claim 1, wherein the means for 
transferring the plurality of reference frames establishes a coupling between a direction 
of visualization of the representation of the non-homogeneous structure on the display 
means and a direction of observation of the non-homogeneous structure and of the 
reference structure by the active intervention member. 

16. The interactive system according to claim 15, further comprising: 

a first module for visualizing a representation in a direction given by two 

points; 

a second module for visualizing a representation in a direction given by an 
angle of elevation and an angle of azimuth. 

17. (Canceled) 

18. (Canceled) 
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19. (Twice Amended) An interactive system for intervention inside a region 
of a patient, said interactive system comprising: 

a device operable to receive image data of the region of the patient, 

wherein the image data includes image data of a first reference structure to establish an 
image reference frame for the region of the patient; 

a second reference structure positioned relative to the patient to establish 

a patient reference frame for the region of the patient; 

a controller operable to correlate the position of the first reference 

structure in the image reference frame with the position of the second reference 
structure in the patient reference frame; 

an active member operable to perform the intervention; and 

a tracking system operable to determine a position of at least the second 

reference structure and a position of the active member and configured to transmit the 
determined positions of the second reference structure and the active member to the 
controller; 

wherein the controller is configured to determine the position of the active 

member based on the determined position of at least the active member and the 
correlation of the first reference structure and the second reference structure. 

20. (previously presented) The interactive system as defined in Claim 19 

wherein the first reference structure includes a plurality of base points. 
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21. (previously presented) The interactive system as defined in Claim 20 

wherein the second reference structure includes a plurality of tracking markers. 

22. (previously presented) The interactive system as defined in Claim 19 

wherein the second reference structure includes a plurality of tracking markers. 

23. (previously presented) The interactive system as defined in Claim 22 

wherein the plurality of tracking markers are attached to the patient 

24. (previously presented) The interactive system as defined in Claim 19 

wherein the second reference structure is attached to the patient. 

25. (previously presented) The interactive system as defined in Claim 19 

wherein the first reference structure is attached to the patient. 

26. (previously presented) The interactive system as defined in Claim 21 

wherein the plurality of base points are generated from the plurality of tracking markers. 

27. (previously presented) The interactive system as defined in Claim 20 

wherein the plurality of base points are at least one of a plurality of notable points on the 
patient and marks fixed to the patient. 
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28. (previously presented) The interactive system as defined in Claim 27 

wherein the notable points are selected from a group comprising a head, eyebrows, 
temples, frontal medial point, an apex of a skull, a center of gravity of an orbits of the 
eves and a combination thereof. 

29. (previously presented) The interactive system as defined in Claim 19 

wherein the tracking system includes a marker device operable to determine a position 
of the second reference structure in relation to the patient reference frame. 

30. (Amended) The interactive system as defined in Claim 29 wherein the 

marker device includes a telemetry system operable to determine the position of the 
second reference structure in the patient reference frame and transmit the determined 
position to the controller wherein the controller is operable to perform the correlation at 
least with the transmitted determined position. 

31 (previously presented) The interactive system as defined in Claim 30 

wherein the telemetry system is an electromagnetic telemetry system. 

32. (previously presented) The interactive system as defined in Claim 31 
wherein the second reference structure includes electromagnetic tracking markers, 
wherein the electromagnetic telemetry system is operable to determine the position of 
the electromagnetic tracking markers of the second reference structure in relation to the 
patient reference frame. 
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33. (previously presented) The interactive system as defined in Claim 32, 

wherein the electromagnetic tracking markers are transmitters and the electromagnetic 
telemetry system is an electromagnetic sensor. 

34. (previously presented) The interactive system as defined in Claim 30 

wherein the telemetry system is an optical telemetry system. 

35. (Amended) The interactive system as defined in Claim 34 wherein the 

optical telemetry system includes at least one of a video camera or an infrared camera 
to image at least the second reference structure and configured to plot points of the 
second reference structure. 

36. (previously presented) The interactive system as defined in Claim 34 
wherein the second reference structure includes optical tracking markers, wherein the 
optical telemetry system is operable to determine the position of the optical tracking 
markers of the second reference structure in relation to the patient reference frame. 

37, (previously presented) The interactive system as defined in Claim 34 

wherein the optical telemetry system utilizes position and shape recognition to identify 
the second reference structure. 

38. (previously presented) The interactive system as defined in Claim 29 

wherein the marker device includes a three-dimensional probe. 
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39. (previously presented) The interactive system as defined in Claim 38 
wherein the three-dimensional probe includes a tactile tip operable to engage the 
second reference structure. 

40. (previously presented) The interactive system as defined in Claim 38 

wherein the three-dimensional probe is roboticallv manipulated, such that the 
instantaneous position of the three-dimensional probe is known. 

41. (previously presented) The interactive system as defined in Claim 29 

wherein the marker device includes a set of cameras operable to determine the position 
of the second reference structure in relation to the patient reference frame. 

42. (previously presented) The interactive system as defined in Claim 41 

wherein the set of cameras are selected from video and infrared cameras. 

43. (previously presented) The interactive system as defined in Claim 29 

wherein the marker device is a laser beam emission system operable to illuminate the 
second reference structure to determine a position of the second reference structure in 
relation to the patient reference frame. 

44. (previously presented) The interactive system as defined in Claim 20 

wherein the controller further includes a graphical tool operable to identify the plurality of 
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base points of the first reference structure in the image data of the image data reference 



frame. 



45. (previously presented) The interactive system as defined in Claim 44 

wherein the graphical tool is a mouse in communication with the controller. 

46. (previously presented) The interactive system as defined in Claim 19 

wherein the first reference structure is generated from the second reference structure. 

47. (canceled) 

48. (Amended) The interactive system as defined in Claim 19 wherein the 

active member is selected from a group comprising a trephining tool, a needle, a laser, 
a radioscope emission head, an endoscopic viewing system, a tool used in the 
intervention, an implant, a sighting system, a microscope, and combinations thereof. 

49. (Amended) The interactive system as defined in Claim 19 further 

comprising a telemetry system operable to determine the position of the active member 
in the patient reference frame, said telemetry system in communication with the 
controller. 
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50. (previously presented) The interactive system as defined in Claim 49 

wherein the position information of the active member is six degree of freedom 
information in relation to the patient reference frame. 

51. (Amended) The interactive system as defined in Claim ff4711 19 wherein 

the device includes a display operable to display the image data of the region of the 
patient in relation to the image reference frame. 

52. (previously presented) The interactive system as defined in Claim 51 

wherein the controller is further operable to determine a reference origin of intervention 
and a direction of intervention and said display is further operable to display the 
reference origin of intervention and direction of intervention. 

53. (previously presented) The interactive system as defined in Claim 51 

wherein the controller is further operable to model a reference origin of intervention and 
a direction of intervention and said display is further operable to display the modeled 
reference origin of intervention and direction of intervention. 

54. (Amended) The interactive system as defined in Claim 51 wherein the 

display is further operable to display the real-time position of the active member in the 
image reference frame based on the determined position of the active member with the 
tracking system. 
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55. (previously presented) The interactive system as defined in Claim 51 

wherein the display is further operable to display image data relative to a direction of 
intervention of the active member. 

56, (previously presented) The interactive system as defined in Claim 55 

wherein the image data is displayed perpendicular to a direction of intervention of the 
active member. 

57. (previously presented) The interactive system as defined in Claim 51 

wherein the controller is further operable to simulate an optimal trajectory of advance of 
the active member and said display is operable to display the optimal trajectory in the 
image data relative to the image reference frame. 

58. (previously presented) The interactive system as defined in Claim 57 

wherein movement of the active member is steered to the optimal trajectory to carry out 
a programmed intervention. 

59. (Amended) The interactive system as defined in Claim 19 wherein the 

active member is roboticallv controlled. 

60. (previously presented) The interactive system as defined in Claim 19 

wherein the image data is at least one of a magnetic resonance image data, a 
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tomographic image data, a radiographic image data, x-rav image data, and 
combinations thereof. 

61. (previously presented) The interactive system as defined in Claim 19 

wherein the device is operable to construct three-dimensional images from captured 
two-dimensional images. 
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62. (previously presented) An interactive system for intervention inside a region 
of a patient, said interactive system comprising: 

a device operable to receive image data of the region of the patient, 

wherein the image data includes image data of a first reference structure to establish an 
image reference frame for the region of the patient; 

a second reference structure positioned relative to the patient to establish 

a patient reference frame for the region of the patient; and 

a controller operable to correlate the position of the first reference 

structure in the image reference frame with the position of the second reference 
structure in the patient reference frame; 

wherein the device is operable to construct three-dimensional images from 

captured two-dimensional images; 

wherein the controller is operable to superimpose two-dimensional image 

data on the three-dimensional images wherein any change in soft external parts of the 
patient can be visualized as compared with the image captured by the imaging device. 
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63. (previously presented) An interactive system for intervention inside a 

region of a patient, said interactive system comprising: 

a device operable to receive image data of the region of the patient 

wherein the image data includes image data of a first reference structure to establish an 
image reference frame for the region of the patient; 

a second reference structure positioned relative to the patient to establish 

a patient reference frame for the region of the patient; 

a controller operable to correlate the position of the first reference 

structure in the image reference frame with the position of the second reference 
structure in the patient reference frame; and 

an active member operable to perform the intervention; 

wherein the device includes a display operable to display the image data 

of the region of the patient in relation to the image reference frame; 

wherein the controller is further operable to determine residual uncertainty 

which is used to represent a contour with dimensions larger than those which would 
normally be represented and the display is operable to display the residual uncertainty 
of the contour. 

64. (previously presented) The interactive system as defined in Claim 63 

wherein the contour is a display of an active member and a representation of residual 
uncertainty in order to reduce the chance of traversing undesired structures. 
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65. (previously presented) The interactive system as defined in Claim 19 

wherein the controller is further operable to correlate map data in a map reference 
frame with the patient reference frame. 

66. (Amended) The interactive system as defined in Claim 19 wherein the 

intervention is at least one of a neurosurgery, orthopedic surgery, cranial surgery, and 
combinations thereof. 

67. (previously presented) The interactive system as defined in Claim 19 

wherein the second reference structure is fixed to a head set. 

68. (previously presented) The interactive system as defined in Claim 60 

wherein the head set is further fixed to an operating table. 

69. (previously presented) The interactive system as defined in Claim 19 

wherein the device further includes memory operable to store the image data. 

70. (previously presented) The interactive system as defined in Claim 19 

wherein the device is a first computer. 

71. (previously presented) The interactive system as defined in Claim 70 

wherein the controller is a second computer. 



Serial No. 09/784,829 



Page 19 of 28 



The interactive system as defined in Claim 71 



72. (previously presented) 



wherein the first computer and the second computer is a single work station. 
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73. (Twice Amended) An interactive system for intervention inside a region 
of a patient, said interactive system comprising: 

a device operable to receive image data of the region of the patient 

wherein the image data includes image data of a first reference structure to establish an 
image reference frame for the region of the patient; 

a second reference structure positioned relative to the patient to establish 

a patient reference frame for the region of the patient; 

a controller operable to correlate the position of the first reference 

structure in the image reference frame with the position of the second reference 
structure in the patient reference frame; 

an active member operable to perform the intervention inside the region of 

the patient; 

a tracking system operable to track the position of the active member in 

relation to the patient reference frame, the tracking system being in communication with 
the controller to transmit the tracked position of the active member as position 
information to the controller, wherein the controller is operable to determine the position 
of the active member relative to the image reference frame; and 

a display operable to display the real-time position of the active member in 

the image reference frame based on the controller determined position of the active 
member based on the tracked position of the active member from the tracking system, 
wherein the controller is configured to generate a representation of the active member 
that is displayed on the display relative to a display of the received image data. 
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74. (previously presented) The interactive system as defined in Claim 73 
wherein the active member is selected from a group comprising a trephining tool, a 
needle, a laser, a radioscope emission head, an endoscopic viewing system, a tool 
used in the intervention, an implant, a sighting system, a microscope, and combinations 
thereof. 

75. (previously presented) The interactive system as defined in Claim 73 

wherein the position information of the active member is six degree of freedom 
information in relation to the patient reference frame. 

76. (previously presented) The interactive system as defined in Claim 

73 wherein the tracking system that tracks the position of the active member is a 
telemetry system in communication with the controller. 

77. (previously presented) The interactive system as defined in Claim 73 

wherein the active member is roboticallv controlled. 

78. (previously presented) The interactive system as defined in Claim 73 

wherein the image data is at least one of a magnetic resonance image data, a 
tomographic image data, a radiographic image data, x-ray image data, and 
combinations thereof. 
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79. (previously presented) The interactive system as defined in Claim 73 

wherein the controller is further operable to determine a reference origin of intervention 
and a direction of intervention and said display is further operable to display the 
reference origin of intervention and direction of intervention. 

80, (previously presented) The interactive system as defined in Claim 73 

wherein the first reference structure includes a plurality of base points, 

81. (previously presented) The interactive system as defined in Claim 80 

wherein the second reference structure includes a plurality of tracking markers. 

82. (previously presented) The interactive system as defined in Claim 81 

wherein the plurality of base points are generated by the plurality of tracking markers. 

83. (previously presented) The interactive system as defined in Claim 73 
wherein the second reference structure is attached to the patient 

84. (previously presented) The interactive system as defined in Claim 73 
wherein intervention is at least one of a neurosurgery, orthopedic surgery, cranial 
surgery intervention, and combinations thereof. 

85. (previously presented) The interactive system as defined in Claim 73 

wherein the second reference structure is fixed to a head set. 
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86. (Amended) The interactive system as defined in Claim 73 wherein the 

display forms part of the device and wherein the image data received is acquired image 
data of the region of the patient and is displayed on the display, further wherein the 
representation of the active member is displayed on the acquired image data of the 
region of the patient 
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87. (Twice Amended) A method for performing an image guided 

intervention inside a region of a patient, said method comprising: 

accessing a first image data of the region of the patient captured with an 

imaging system where the first image data includes image data of a first reference 
structure; 

identifying the first reference structure in the first image data to establish 

an image reference frame; 

identifying a second reference structure relative to the patient to establish 

a patient reference frame; 

correlating the position of the first reference structure in the image 

reference frame in the first image data with the position of the second reference 
structure in the patient reference frame; and 

tracking an active member at least to determine a position of the active 

member in the patient reference frame to determine a location of the active member 
based on the tracking of the active member and transmitting the determined position in 
the patient refrence frame for display on a display device relative to the image reference 
frame of the first image data based at least on the correlation of the first reference 
structure and the second reference structure. 

88. (previously presented) The method as defined in Claim 87 further 

comprising attaching a plurality of tracking markers to the patient where the tracking 
markers form the second reference structure. 
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89, (previously presented) The method as defined in Claim 88 further 

comprising identifying the position of the tracking markers in the patient reference frame 
using a telemetry system. 

90, (Amended) The method as defined in Claim 89 further comprising 

transmitting from the tracking markers a signal and receiving the transmitted signal with 
an electromagnetic sensor to identify the position of the second reference structure in 
the patient reference frame. 

91. (previously presented) The method as defined in Claim 87 wherein 

identifying the first reference structure includes identifying a plurality of base points 
visible in the image data. 

92. (previously presented) The method as defined in Claim 91 wherein 

identifying the plurality of base points includes identifying at least one of notable points 
on the patient as marks fixed to the patient representing the plurality of base points. 

93. (previously presented) The method as defined in Claim 92 wherein the 

notable points are selected from a group comprising a head, eyebrows, temporal point, 
frontal medial point, an apex of a skull, a center of gravity of an orbits of the eves and a 
combination thereof. 
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94. (previously presented) The method as defined in Claim 91 wherein the 

plurality of base points visible in the image data are generated from the plurality of 
tracking markers attached to the patient. 

95. (previously presented) The method as defined in Claim 87 further 

comprising attaching the second reference structure to the patient. 

96. (previously presented) The method as defined in Claim 87 further 

comprising displaying the image data of the region of the patient, including displaying 
the first reference structure. 

97. (previously presented) The method as defined in Claim 87 further 

comprising performing an intervention on the patient with an active member. 

98. (Canceled) 

99. (Amended) The method as defined in Claim 96 further comprising: 

displaying the position of the active member as a representation of the 

active member in the accessed first image data that is captured image data that is 
correlated to the patient based on the correlation and displayed on a display device with 
the position of the active member being correlated between the patient reference frame 
defined by the first reference structure fixed to the patient and the image reference 
frame based on the tracking of the active member. 
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100. (Amended) The method as defined in Claim 99 further comprising 

identifying the position of the active member with a telemetry system by transmitting the 
tracked location of the active member for displaying the representation of the active 
member. 

101. (previously presented) The method as defined in Claim 99 further 

comprising displaying a reference origin of intervention and a direction of intervention in 
the image data. 

102, (previously presented) The method as defined in Claim 101 further 

comprising tracking the position of the active member relative to the reference origin of 
intervention and the direction of intervention. 

103. (previously presented) The method as defined in Claim 87 further 

comprising converting two-dimensional image data to three-dimensional image data. 

104. (previously presented) The method as defined in Claim 97 wherein the 

intervention is selected from at least one of a neurosurgery, orthopedic surgery, cranial 
surgery, and combinations thereof. 

105. (previously presented) The method as defined in Claim 95 further 

comprising attaching the second reference structure to a head set. 
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ABSTRACT 



An interactive system for a local intervention inside a region 
of a non-homogeneous structure, such as the skull of a 
patient, which is related to the frame of reference (R^ of an 
operation table, and which is connected to a reference 
structure comprising a plurality of base points. The system 
creates on a screen a representation of the non-homogeneous 
structure and of the reference structure connected thereto, 
provides the coordinates of the images of the base points in 
the first frame of reference (R^), allows the marking of the 
coordinates of the base points in R 2 , and allows the carrying 
out of the local intervention with an active member such as 
a trephining tool, a needle, or a radioactive or chemical 
implant. The systems also optimizes the transfer of reference 
frames between Rj and R 2 , from the coordinates of the base 
points in R 2 and the images in R a by reducing down to a 
minimum the deviations between the coordinates of images 
in R a and the base points in R a after transfer. The system also 
establishes real time bi-directional coupling between: (1) an 
origin and a direction of intervention simulated on the 
screen, (2) the position of the active member. 

16 Claims, 13 Drawing Sheets 




5,868,675 

Page 2 



OTHER PUBLICATIONS 

Batnitzky, S. et al. Three-Dimensional Computer Recon- 
struction from Surface Contours for Head CT Examinations. 
J. Comp. Asst. Tomogr., vol. 5, No. 1, pp. 60-67, Feb. 1981. 
Roberts, D.W., et al. A Frameless Stereotaxic Integration of 
Computerized Tomographic Imaging and the Operating 
Microscope. J. Neurosurg., vol. 65, pp. 545-549, 1986. 
Kelly, P.J. et al. Computer-assisted Stereotactic Laser 
Microsurgery for the Treatment of Intracranial Neoplasms, 
Neurosurgery, vol 10, No. 3, pp, 324-330, 1982. 
S. Lavallee, "A New System for Computer Neurosurgery", 
IEEE Eng. in Medicine & Bio. Soc, 11th Annual Int. Conf., 
Nov, 9-12, 1989, pp. 926-927. 



Watanabe et al., Three Dimensional Digitizer (Neuronavi- 
gator): New Equipment for Computed Tomography-Guided 
Stereo toxic Surgery, Surg. Neurol., 1987, No. 27, pp. 
543-547. 

PJ. Kelly et al., "Computer-Assisted Stereotactic Laser 
Microsurgery for the Treatment of Intracranial Neo- 
plasms", Neuro., vol. 10, No. 3, 1982, pp, 324-330. 
Batnitzky et al., "Three-dimensional Computer Reconstruc- 
tion From Surface Contours for Head CT Examinations", 
Journal of Comp. Assisted Tomography, No. 5, Feb. 1981, 
pp. 60-67 

Roberts et al., "A Frameless Stereotoxic Integration of 
Computerized Tomographic Imaging and the Operating 
Microsc gsl", J. of Neuro. Surg. No. 65, 1986, pp. 545-549. 



U.S. Patent 



Feb. 9, 1999 Sheet 1 of 13 



5,868,675 




U.S. Patent 



Feb. 9, 1999 Sheet 3 of 13 



5,868,675 




U.S. Patent Feb. 9, 1999 Sheet 4 of 13 



5,868,675 



2D iroa^e data. 



,100 



Selection of structures? of interest. / 



101 



Generation of the 2>D database / 



102 



C&lt cad tjpe aoPtware 



f 



103 



High resolution screen display oF: 

Re presentation oFLVie nonhomo^eneoa$ 
Structure^ 

and oF the reference structure 



StmuLa.h\on oF intervention pa.th 
on £D and 3D images 



,105 




FIG.5a 



U.S. Patent 



Feb. 9, 1999 Sheet 5 of 13 



5,868,675 



Continue 1 



107 

J 



of base points 
With respect fco 
the &ecor\d 

reference Frame 



106 
L 



Markm^oFthe'trnageS oP 
the base pom te> with res- 
pect to the. first refe - 
-rence fr<ame 



108- 



Eta-torabton of the. toots 
for reference frame 
transfer and transfer 
of orncyn 





Intervention ; 
Steering or the boot 
ov^tr the s*rnuLa.teci 
intervention path 


2>D pLot oF^posi- 
-tton o^r" the 
tnberv/enbton tooL 







109 

* Test the A 
points 



FIG.5b 



U.S. Patent 



Feb. 9, 1999 Sheet 6 of 13 



5,868,675 



1001 



Acc^o\£>it>©n of khe 3 reference points, 
A^yC^on the repreaentab\An / expre&&ed \o bhe 
^LobaL basis by section ontheSD 
^eonnebric nnodLeL 



1002 



Creation of a -3* 3 orthogonal, unit 
characteristic oFa rf^hb -handed 

OrthonormaL t>as»\& (" L ^ k ) 



1003 

u 



I 



Accjui&ikton of the. 2> reParence points 

D,E,r on the PATIENT: 

e^prcaseci "in the gLobal. ba&\& by 3D 



100^ 
U 



Creation oFa 3^3 orthogonal- unit matrix N 
Characteristic oF <a ri^ht ~ha.n<=Jeci 
orLhonormaL ba*s\s {i\ J f Y*) 



VaUdah\on: choice, of" matnees M N after 
besting additional, base point (s) 



FIG 6 



U.S. Patent 



Feb. 9, 1999 Sheet 7 of 13 



5,868,675 



2010 



Aceyj\s\t'»On of the point to be. tran&fia.rreei Ofi 
the repre&entation^XM^yM^M) b^> 
selection on the. 3D geometric. mocieL 



2011 



CaUcuLatvon or Line coords nah&s (XP ; yf? ZP) 
on the pafc.Ie.nt bnroo^h tbe. tran&f&rmatlon 



Procese^n^ outvie coordinates £xP ; yf}ZP) 



201 



riG-7 



U.S. Patent 



Feb. 9, 1999 Sheet 8 of 13 



5,868,675 



r 



Acquisition oF the, point to be transferred 
on the pat'tcnt^P, yP,ZP) 3D ^^12.103 



T 



aoao 



C&LcuLat'ion oF tme cooreiicia±.e<2p(XM yH ; ZM) 
on the represe.nta.ttOn through k-Vie. 

CXM,YM,ZttK N*M" *Cxp,yp 1 zp) 



^021 



DispLap, of the. point. (XM^f^/ZM) h>y th< 
insuLabi^atvon system 



202a 

._■ I 



-202 



riG_8 



U.S. Patent 



Feb. 9, 1999 Sheet 9 of 13 



5,868 



,675 



|30K) 



Ac^oi&fbion oFbhe 2. points A,B for the direction as» 

"\coo<"ci\oab<s-S, rcpreaenbabion fcy 3 Ddieyk.iz.in^ 

or, the patient a.r\d bran&fJtr or s>eLtcfc\on 
On the. 3D^eometric model- 



3011 



Cr&a\z.\ori of a 3*3 un\t ortho^onaL matrix 
characteristic. oFa right-handed orthonorrnat 



301 



V 



For <2>LL reference poinhs oP the. «snt'itve& 
r>om the 3D database 
(Xw ; Xw,2-v/) 



3015 



~~m output 



CaLcuLatioo oF the coordinates oF the point- 
in the bas>i& ( 1" j 1 **, K 11 ) through Lhe teansfbema- 
-Lion 



30V* 



Dts>pLa^ oF the ^pLane. 



riG.9a 



U.S. Patent Feb. 9, 1999 Sheet 10 of 13 5,868,675 



I 

.3020 



Accp&ihionoFkhe fc-vjo an 3 Les For the d.Yectiorv. 
\J elevation and a^imuthin the reprefee.otafc.ior 
^arne oF" reference- 



Character! &b<o oF a r\ ^LW -handed ocVhoAowaU 



3022 



3025 



For alt reFcrcoce pombe> oFLhe 
er»hik»k^ From hhts 30 database 



OUTPUT 



3023 



CaLcuLabion oF'th* coorJinabes oFkhe pe >\nt \n 
the. t,a&\s(t'" ( j' , ;k"') through the. transformation 



3024 



Z. 



D*it>pL*y a F the t j- plane 



302 



r!G.9b 



U.S. Patent 



Feb. 9, 1999 



Sheet 11 of 13 



5,868 



Data Uoadiin^ 



4000 



Parameterization oF fc.be Q?e$ acaLe£» 
PLaee \r\ orcier 

CaLibrabion [image a-ndi F (Frame) 



4001 



Generation o^the globat vieW^o^h-vjiew) / 4002 



Automatic cf*febr!bot\on orsect\<5n6<?n bhe 
work Screen 



/ 



4003 



Manual seLecklon oF Sections. 



/4004 



Choice <s?F ehrakew^y (entoj, target) 
hor^or» baL / j&a<^btaL ; FconbaL s<2ctions 



/ 



DlfePLAy oFfce-bfci'^e, oT-STEREOTA^C FRAME y 



A006 



'"Continue A 



FIGJOa 



U.S. Patent Feb. 9, 1999 Sheet 12 of 13 5,868,675 



FlGJOb 












■ • * prot^s- 



**008 



.Search For jfetral 
pabient 


le-Ojic. po\nb& on the. 


\ 






i 


registration o£~cjOi£ 
Op S>k\o Arvdi bOAe 


ie£? before opening I 
FLap I 



t+cno 



ion 



Inbc^r v/ention 



U.S. Patent 



Feb. 9, 1999 



Sheet 13 of 13 



5,868,675 



n 



Starb 




figjoc 



be.Lecb.ion of" eLerr\e,<*v ba 
SbrucbureS oF~ vo>t<~c-e.S>b 



5000 



hodeLi^ pF the structure 
oF i ohere. & b. 



r 



5001 



3D KTia^ing Su-nuLabionorWoe ^5002 
<dUr<sc.bi©«-it> op a-Jm 



4 



bV\«. onarKe> P\4 ko H4 



5003 



Saarch For the po&ttion oFbtw.y--500A. 



I 



pabieob S>pace5^abnV beans- 



5005 



Cou pb\nc^ 



5006 



booL SO WibV^ nrvo^e^fr\etr\b£> over y~ 5007 
bWa, StmuLabcd path 



CatL up div&pLa^ bvwo- 
clirr\£,n£>\oriaL or^rxaL images 



/■ 



5006 



Data. 



3 



/ 



5009 



5,868,1 

1 

INTERACTIVE SYSTEM FOR LOCAL 

INTERVENTION INSIDE A 
NONHUMOGENEOUS STRUCTURE 

The invention relates to an interactive system for local 5 
intervention inside a region of a nonhomogeneous structure. 

The performing of local interventions inside a nonho- 
mogeneous structure, such as intracranial surgical opera- 
tions or orthopedic surgery currently poses the problem of 
optimizing the intervention path or paths so as to secure, on 
the one hand, total intervention over the region or structure 
of interest, such as a tumor to be treated or explored and, on 
the other hand, minimal lesion to the regions neighboring or 
adjoining the region of interest, this entailing the localizing 
and then the selecting of the regions of the nonhoraogeneous 
structure which arc least sensitive to being traversed or the 15 
least susceptible to damage as regards the integrity of the 
structure. 

Numerous works aimed at providing a solution to the 
aboveraentioned problem have hitherto been the subject of 
publications. Among the latter may be cited the article 20 
entitled "Three Dimensional Digitizer (Neuronavigator): 
New Equipment for computed Tomography Guided Stereo- 
taxic Surgery", published by Eiju Watanabe, M.D., Takashi 
Watanabe, M.D., Shinya Manaka, M.D., Yoshiaki 
Mayanagi, M.D., and Kintomo Takakura, M.D. Department 25 
of Neurosurgery, Faculty of Medicine, University of Tokyo, 
Japan, in the journal Surgery Neurol. 1987: 27 pp. 543-547, 
by Elsevier Science Publishing Co., Inc. The Patent WO-A- 
88 09151 teaches a similar item of equipment. 

In the abovementioned publications are described in 30 
particular a system and an operational mode on the basis of 
which a three-dimensional position marking system, of the 
probe type, makes it possible to mark the three-dimensional 
position coordinates of a nonhomogeneous structure, such as 
the head of a patient having to undergo a neurosurgical 35 
intervention, and then to put into correspondence as a 
function of the relative position of the nonhomoc.eneous 
structure a series of corresponding images consisting of 
two-dimensional images sectioned along an arbitrary 
direction, and obtained previously with the aid of a medical 40 
imaging method of the "scanner" type. 

The system and the operational mode mentioned above 
offer a sure advantage for the intervening surgeon since the 
latter has available, during the intervention, apart from a 
direct view of the intervention, at least one two-dimensional 45 
sectional view enabling him to be aware, in the sectional 
plane, of the state of performance of the intervention. 

However, and by virtue of the very design of the system 
and of the operational mode mentioned above, the latter 
allow neither a precise representation of the state of perfor- 50 
mance of the intervention, nor partially or totally automated 
conduct of the intervention in accordance with a program for 
advance of the instrument determined prior to the interven- 
tion. 

Such a system and such an operational mode cannot 55 
therefore claim to eradicate all man-made risk, since the 
intervention is still conducted by the surgeon alone. 

The objective of the present invention is to remedy the 
whole of the problem cited earlier, and in particular to 
propose a system permitting as exact as possible a 60 
correlation, at any instant, between an intervention modeling 
on the screen and the actual intervention, and furthermore 
the representation from one or more viewing angles, and if 
appropriate in one or more sectional planes, of the nonho- 
mogeneous structure, the sectional plane or planes possibly 65 
being for example perpendicular to the direction of the path 
of advance of the instrument or of the intervention tool. 
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Another objective of the present invention is also the 
implementation of a system permitting simulation of an 
optimal trajectory of advance of the tool, so as to constitute 
an assisted or fully programed intervention. 

Finally, an objective of the present invention is to pro- 
pose a system making it possible, on the basis of the 
simulated trajectory and of the programed intervention, to 
steer the movement of the instrument or tool to the said 
trajectory so as to carry out the programed intervention. 

The invention proposes to this effect an interactive 
system for local intervention inside a region of a nonhomo- 
geneous structure to which is tied a reference structure 
containing a plurality of base points, characterized in that it 
comprises: 

means of dynamic display by three-dimensional imaging 
of a representation of the nonhomogeneous structure 
and of a reference structure tied to the nonhomoge- 
neous structure, including images of the base points, 
means of delivering the coordinates of the images of the 

base points in the first reference frame, 
means of securing the position of the non-homogeneous 
structure and the reference structure with respect to a 
second reference frame, 
marker means for delivering the coordinates of the base 

points in the second reference frame, 
means of intervention comprising an active member 
whose position is determined with respect to the second 
reference frame, 
means of optimizing the transfer of reference frames from 
the first reference frame to the second reference frame 
and vice versa, on the basis of the coordinates of the 
images of the base points in the first reference frame 
and of the coordinates of the base points in the second 
reference frame, in such a way as to reduce to a 
minimum the deviations between the coordinates of the 
images of the base points in the first reference frame 
and the coordinates of the base points, expressed in the 
said first reference frame with the aid of the said 
reference frame transfer tools, 
means for defining with respect to the first reference 
frame a simulated origin of intervention and a simu- 
lated direction of intervention, and 
reference frame transfer means using the said reference 
frame transfer tools to establish a bidirectional coupling 
between the simulated origin of intervention and the 
simulated direction of intervention and the position of 
the active member. 
A more detailed description of the system of the invention 
will be given below with reference to the drawings in which; 

FIG. 1 represents a general view of an interactive system 
for local intervention inside a region of a nonhomogeneous 
structure according to the present invention, 

FIG. 2 represents, in the case where the nonhomogeneous 
Structure consists of the head of a patient, and with a view 
to a neurosurgical intervention, a reference structure tied to 
the nonhomogeneous structure and enabling a correlation to 
be established between a "patient" reference frame and a 
reference frame of images of the patient which were made 
and stored previously, 

FIG. 3 represents an advantageous embodiment of the 
spacial distribution of the reference structure of FIG. 2, 

FIG. 4 preesents an advantageous embodiment of the 
intervention means set up on an operating table in the case 
of a neurosirgical intervention, 

FIGS. 5a and 5b represent a general flow diagram of 
ftmcrional steps implemented by the system, 
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FIGS. 6 thru 8 represent flow diagrams of programs transfer using on the one hand the coordinates in R 2) 

permitting implementation of certain functional steps of measured by the probe 3, of a plurality of base points of the 

FIG, 56, structure SR, and on the other hand the coordinates in R 1( 

FIG. 9a represents a flow diagram of a program permit- determined by graphical tools of the computer M01 

ting implementation of a functional step of FIG. 5a, 5 (pointing by mouse, etc.), of the images of the corresponding 

FIG. 9b represents a flow diagram of a program permit- base points in the representation RSR, so as to secure the 

ting implementation of another functional step of FIG. 5a, best possible correlation between the information modeled 

FIGS. 10*7 and 10£> represent a general flow diagram of in the computer equipment and the corresponding real-world 

the successive steps of an interactive dialogue between the information. 

system of the present invention and the intervening surgeon 10 There is furthermore provision for reference frame trans- 

and fer means 11 designed to use the tools thus elaborated and 

FIG, 10c represents a general flow diagram of the sue- to secure this correlation in real time, 

cessive functional steps carried out by the system of the Moreover, means 40 are provided, as will be seen in detail 

invention, having (sic) the intervention, prior to the later, for determining or modeling a reference origin of 

intervention, during the intervention and after the interven- 15 intervention ORI and a direction of intervention A. 

tion. With the aid of the means 11, the modeled direction of 

The interactive system for local intervention according to intervention A can, at least prior to the intervention and at the 

the invention will firstly be described in connection with start of the intervention, be materialized through an optical 

FIG. 1. sighting system available to the surgeon, it being possible to 

A nonhomogeneous structure, denoted SNH, on which an 20 steer this sighting system positionally with respect to the 

intervention is to be performed, consists for example of the second reference frame R 2 . 

head of a patient in which a neurosurgical intervention is to The sighting system will be described later, 

be performed. It is however understood that the system of The system of the present invention finally comprises 

the invention can be used to carry out any type of interven- means 5 of intervention comprising an active member, 

lion in any type of nonhomogeneous structure inside which 25 denoted 50, whose position is specified with respect to the 

structural and/or functional elements or units may be in second reference frame R 2 . The active member can consist 

evidence and whose integrity, during the intervention, is to of the various tools used in surgical intervention. For 

be respected as far as possible. example, in the case of an intercranial neurosurgical 

The system comprises means, denoted 1, of dynamic intervention, the active member could be a trephining tool, 

display by three-dimensional imaging, with respect to a first 30 a needle, a laser or radioscope emission head, or an endo- 

reference frame R u of a representation (denoted RSR) of a scopic viewing system. 

reference structure SR (described later) tied to the structure According to an advantageous characteristic of the 

SNH, and a representation or modeling of the nonhomoge- invention, by virtue of the reference frame transfer means 

neous structure, denoted RSNH. 11, the position of the active member can be controlled 

More precisely, the means 1 make it possible to display a 35 dynamically on the basis of the prior modeling of the origin 

plurality of successive three-dimensional images, from dif- of intervention ORI and of the direction of intervention A. 

ferent angles, of the representations RSNH and RSR. The means 1 of dynamic display by three-dimensional 

The system of the invention also comprises means, imaging of the representations RSNH and RSR comprise a 

denoted 2, of tied positioning, wiLh respect to a second file 10 of two-dimensional image data. The file 10 consists 

reference frame R 2 , of the structures SNH and SR. 40 for example of digitized data from tomographic sections, 

In the present non- limiting example, the head of the from radiographs, from maps of the patient's head, and 

patient, bearing the reference structure SR, is fixed on an contained in an appropriate mass memory, 

operating table TO to which are fixed the means 2 of tied The successive tomographic sections can be produced 

positioning. prior to the intervention in a conventional manner, after the 

Of course, the patient whose head has been placed in the 45 reference structure SR has been put in place on the nonho- 

means 2 for tied positioning has previously been subjected mogeneous structure SNH. 

to the customary preparations, in order to enable him to According to an advantageous feature, the reference 

undergo the intervention. structure SR can consist of a plurality of marks or notable 

The means 2 of the tied positioning with respect to R 2 will points which can be both sensed by the marker means 3 and 

not be described in detail since they can consist of any 50 detected on the two-dimensional images obtained, 

means (such as a retaining headset) normally used in the Of course, the abovementioned two-dimensional tomo- 

field of surgery or neurosurgery. The reference frame R 2 can graphic sections can likewise be produced by any medical 

arbitrarily be defined as a tri-rectangular reference trihedron imaging means such as a nuclear magnetic resonance sys- 

tied to the operating table TO, as represented in FIG. 1. tern. 

Means 3 of marking, with respect to the second reference 55 In a characteristic and well-known manner, each two- 
frame R 2 , the coordinates, denoted X2, Y2, Z2, of arbitrary dimensional image corresponding to a tomographic scanner 
points, and in particular of a certain number of base points section corresponds to a structural slice thickness of about 2 
of the reference structure SR are furthermore provided. to 3 mm, the pixels or image elements in the plane of the 

These base points constituting the reference structure SR tomographic section being obtained with a precision of the 

can consist of certain notable points and/or of marks fixed to 60 order of ±1 mm. It is therefore understood that the marks or 

the patient, at positions selected by the surgeon and in points constituting the reference structure SR appear on the 

particular at these notable points. images with a positional uncertainty, and an important 

The system of the invention further comprises computing feature of the invention will consist in minimizing these 

means 4 receiving means 3 of marking the coordinates X2, uncertainties as will be described later. 

X2, 22. 65 The system also comprises first means 110 for calculating 

The computing means 4, as will be seen in detail later, are and reconstructing three-dimensional images from the data 

designed to elaborate optimal tools for reference frame from the file 10. 
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It also comprises a high-resolution screen 12 permitting 
the displaying of one or more three-dimensional or two- 
dimensional images constituting so many representations of 
the reference structure RSR and of the nonhomogeneous 
structure SNH. 

Advantageously, the calculating means 110, the high- 
resolution screen and the mass memory containing the file 
10 form part of a computer of the workstation type with 
conventional design and denoted MOL 

Preferably, the first calculating means 110 can consist of 
a CAD type program installed in the workstation MOl. 

By way of non-limiting example, this program can be 
derived from the software marketed under the tradename 
"AUTOCAD" by the "AUTODESK" company in the 
United States of America. 

Such software makes it possible, from the various digi- 
tized two-dimensional images, to reconstruct threedimen- 
sional images constituting the representations of the struc- 
tures RSR and RSNH in arbitrary orientations. 

Thus, as has furthermore been represented in FIG. 1, the 20 
calculating means 4 and 11 can consist of a third computer, 
denoted M02 in FIG. 1. 

The first and second computers MOl and M02 are 
interconnected by a conventional digital link (bus or 
network). 25 

As a variant, the computers MOl and M02 can be 
replaced by a single workstation. 

The marker means 3 consist of a three-dimensional probe 
equipped with a tactile tip 30. 

This type of three-dimensional probe, known per se and 30 
not described in detail, consists of a plurality of hinged arms, 
marked in terms of position with respect to a base integral 
with the operating table TO. It makes it possible to ascertain 
the coordinates of the tactile tip 30 with respect to the origin 
0 2 of the reference frame R 2 with a precision better than 1 35 
mm. 

The probe is for example equipped with resolvers deliv- 
ering signals representing the instantaneous position of the 
abovementioned tactile Lip 30. The resolvers are themselves 
connected to the circuits for digital/analog conversion and 40 
sampling of the values representing these signals, these 
sampling circuits being interconnected in conventional man- 
ner to the second computer M02 in order to supply it with 
the coordinates X2, X2, Z2 of the tactile tip 30. 

As a variant or additionally, and as represented 45 
diagraramatically, the marker means 3 can comprise a set of 
video cameras 31 and 32 (or else infrared cameras) enabling 
pictures to be taken of the structures SNH and SR. 

The set of cameras can act as a stereoscopic system 
permitting the positional plotting of the base points of the 50 
reference structure SR, or of other points of the nonhomo- 
geneous structure SNH, with respect to the second reference 
frame R 2 . The positional plotLing can be done for example 
by appending a laser beam emission system making it 
possible to illuminate successively the points whose coor- 55 
dinates are sought, appropriate software making it possible 
to then determine the position of these points one by one 
with respect to R 2 , This software will not be described since 
it can consist of position and shape recognition software 
normally available on the market. 60 

According to another variant, the marker means 3 can 
comprise a telemetry system. 

In this case, the marks of the structure SR can consist of 
small radiotransmitters implanted for example on the rel- 
evant points of the patient's head and designed to be visible 65 
on the two-dimensional images, appropriate electromagnetic 
or optica] sensors (not shown) being provided in order to 
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determine the coordinates of the said marks in the reference 
frame R 2 or in a reference frame tied to the latter. 

It is important to note here that the general function of the 
base points of the reference structure is, on the one hand, to 
5 be individually localizable on the reference structure, in 
order to deduce from this the coordinates in R 2 , and on the 
other hand, to be visualizable on the two-dimensional 
images so as to be identified (by their coordinates in Rj) and 
included in the representation RSR on the screen. 
10 It can therefore involve special marks affixed at arbitrary 
points of the lateral surface of the structure SNH, or else at 
notable points of the latter, or else, when the notable points 
can in themselves be localized with high precision both on 
the structure SNH and on the 2D sections, notable points 
15 totally devoid of marks. 

In FIG. 2 a plurality of marks, denoted Ml to Mi, these 
marks, in the case where the nonhomogeneous structure 
consists of the head of a patient, being localized for example 
between the eyebrows of the patient, on the latter* s temples, 
and at the apex of the skull at a notable point such as the 
frontal median point. 

More generally, for a substantially ovoid volume consti- 
tuting the nonhomogeneous structure, there is advanta- 
geously provision for four base points at least on the outer 
surface of the volume. 

Thus, as has been represented in FIG. 3, the four marks 
Ml to M4 of the reference structure are distributed so as 
preferably to define a more or less symmetric tetrahedron. 
The symmetry of the tetrahedron, represented in FIG. 3, is 
materialized by the vertical symmetry plane PV and the 
horizontal symmetry plane PH. 

According to an advantageous characteristic, as will be 
seen later, the means of elaborating the reference frame 
transfer tools are designed to select three points of the 
tetrahedron which will define the "best plane" for the 
reference frame transfer. 

Also, the presence of four or more points enables the 
additional point(s) to validate a specified selection. 

More precisely, the presence of a minimum of four base 
points on the reference structure makes it possible to search 
for the minimum distortion between the points captured on 
the patient by the marker means consisting for example of 
the three-dimensional probe and the images of these points 
on the representation by three-dimensional imaging, the 
coordinates of which are calculated during processing. The 
best plane of the tetrahedron described earlier, that is to say 
the plane for which the uncertainty in the coordinates of the 
points between the points actually captured by the three- 
dimensional probe and the points of the representation of the 
reference structure RSR, is minimal, then becomes the 
reference plane for the reference frame transfer. Thus, the 
best correlation will be established between a modeled 
direction of intervention and a modeled origin of 
intervention, on the one hand, and the action of the member 
50. Preferably, the origin of intervention will be placed at the 
center of the region in which the intervention is to be carried 
out, that is to say a tumor observed or treated for example. 

Furthermore, it will be possible to take the noted residual 
uncertainty into account in order to effect the representation 
of the model and of the tools on the dynamic display means. 

A more detailed description of the means of intervention 
5 will now be given in connection with FIG. 4. 

Preferably, the means of intervention 5 comprise a car- 
riage 52 which is translationally mobile along the operating 
table TO, for example on a rack, denoted 54, whilst being 
driven by a motor, not shown, itself controlled by the 
computer M02 for example, via an appropriate link. This 
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movement system will not be described in detail since it appropriate, make it possible to record the path of the 
corresponds to a conventional movement system available sighting direction, constituting in particular the actual direc- 
on the market. As a variant, the carriage 52 can be mobile tion of intervention, on the representation of the nonhomo- 
ovcr a distinct path separated from the operating table TO, geneous structure in the dynamic display means 1. 
or immobile with respect to the operating table and then 5 Furthermore, as will be described later and in preferential 
constitute a support. manner, the intervening surgeon will be able firstly to define 
The support carriage 52 comprises in the first place a a simulated intervention path and steer thereto the move- 
sighting member OV, constituting the above-mentioned mcnis of . £he active member 50 in the nonhomogeneous 
sighting system, which can consist of a binocular telescope. structure m order effectively to secure all or part of the 

The sighting member OV enables the surgeon, prior to the 10 10 ^yen ion. . t 

. , . , , . i t , f • L lt . In this case, the progress of the intervention tool 50 is then 

actual intervention or during the latter, to sight the pre- directl £ £ h ^ A) 

sumea position 01 tne region in wmch the intervention is to jnvolving l[]6 r ; f(jrence frame 

means 11 in order to 

be earned out. ^ express the path in the reference frame R 2 . 

Furthermore, and in a non-limiting manner, with the A more detailed description of the implementation of the 

sighting member OV can be associated a helium-neon laser 15 operational mode of the system of the invention will now be 

emission system, denoted EL, making it possible to secure described in connection with FIGS. 5a and 56. 

the aiming of a fine positioning or sighting laser beam on the According to FIG. 5a, the first step consists in obtaining 

structure SNH and in particular, as will be seen in detail and organizing in memory the two-dimensional image data 

later, to indicate to the surgeon the position of an entry point (step 100). Firstly, the nonhomogeneous structure SNH is 

PE prior to the intervention, to enable the latter to open the 20 prepared. In the case of a neurosurgical intervention for 

skull at the appropriate location, and likewise to indicate to example, this means that the patient's head can be equipped 

him what the direction of intervention will be. Additionally, with marks constituting the base points of the reference 

the illuminating of the relevant point of the nonhomoge- structure SR. These marks can be produced by means of 

neous structure or at the very least the lateral surface of the points consisting of a dye partially absorbing the X-rays, 

latter enables the video cameras 31 and 32 to carry out, if 25 such as a radiopaque dye. 

necessary, a positional plotting. The abovementioned marks are implanted by the surgeon 

Preferably, a system for measuring position by telemetry on the patient's head at notable points of the latter [sic], and 

53 is provided to secure the precise measurement of the images can then be taken of the nonhomogeneous structure 

position of the support carriage 52 of the sighting member SNH by tomography for example, by means of an apparatus 

OV and of the laser emission system EL. During the 30 of the X-ray scanner type. 

operation, and in order to secure the intervention, the This operation will not be described in detail since it 

carriage 52 can be moved along the rack 54, the position of corresponds to conventional operations in the field of medi- 

the carriage 52 being measured very precisely by means of cal imaging. 

the system 53. The telemetry system 53 is interconnected The two-dimensional image data obtained are then con- 

with the microcomputer M02 by an appropriate link. 35 stituted as digitized data in the file 10, these data being 

The means of intervention 5 can advantageously consist themselves marked with respect to the reference frame R a 

of a guide arm 51 for the active member 50. and making it possible, on demand, to restore the two- 

The guide arm 51 can advantageously consist of several dimensional images onto the dynamic display means 1, 

hinged segments, each hinge being equipped with motors these images representing superimposed sections of the 

and resolvers making it possible to secure control of move- 40 nonhomogeneous structure SNH. 

ment of the end of the support arm and the positional From the digitized image data available to the surgeon, 

plotting of this same end and therefore of the active member the latter then proceeds, as indicated at 101 in FIG. 5a, to 

50 according to six degrees of freedom with respect to the select the structures of interest of the abovementioned 

carriage 52. The six degrees of freedom comprise, of course, images. 

three translalional degrees of freedom with respect to a 45 The purpose of this step is to facilitate the work of the 
reference frame tied to the carriage 52 and three rotational surgeon by forming three-dimensional images which con- 
degrees of freedom along these same axes. tain only the contours of the elements of the structure which 

Thus, the support arm 51 and the member 50 are marked are essential for geometrical definition and real-time raoni- 

in terms of instantaneous position with respect to the second toring of the progress of the intervention, 

reference frame R 2 , on the one hand by way of the positional 50 In the case where the nonhomogeneous structure SNH 

plot of the mobile carriage 52 and, on the other hand, by way consists of the head of a patient, an analysis of the two- 

of the resolvers associated with each hinge of the support dimensional image data makes it possible, from values of 

arm 51. optical density of the corresponding image-points, straight- 

In the case of an intracranial neurosurgical surgical away to extract the contours of the skull, to check the 

intervention, the active member 50 can be removed and can 55 distance scales, etc. 

consist of a trephining tool, a needle or radioactive or Preferably, the abovementioned operations are performed 

chemical implant, a laser or radioisotope emission head or on a rectangle of interest for a given two-dimensional image, 

an endoscopic viewing system. These various members will this making it possible, by moving the rectangle of interest, 

not be described since they correspond to instruments nor- to cover the whole of the image. 

rnally used in neurosurgery. 60 The above analysis is performed by means of suitable 

The materializing of the modeled direction of intervention software which thus makes it possible to extract and vec- 

can be effective by means of the laser emitter EL. This torize the contours of the structures which will be modeled 

sighting being performed, the guide arm 51 can then be in the representations RSNH and RSR. 

brought manually or in steered manner into superposition The structures modeled in the case of a neurosurgical 

with the direction of intervention A. 65 intervention are for example the skull, the cerebral 

In the case of manual positioning, the resolvers associated ventricles, the tumor to be observed or treated, the falx 

with the sighting member OV and the laser emitter EL, if cerebri, and the various functional regions. 
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According to a feature of the interactive system of the 
invention, the surgeon may have available a digitizing table 
or other graphics peripheral making it possible, for each 
displayed two-dimensional image, to rectify or complete the 
definition of the contour of a particular region of interest. 5 

It will be noted finally that by superimposing the extracted 
contours on the displayed two-dimensional image, the sur- 
geon will be able to validate the extractions carried out. 

The extracted contours are next processed by sampling 
points to obtain their coordinates in the reference frame R i , 
it being possible to constitute these coordinates as an ASCII 
type file. This involves step 102 for generating the three- 
dimensional data base. 

This step is followed by a step 103 of reconstructing the 
three-dimensional model This step consists firstly, with the 
aid of the CAD type software, in carrying out on the basis 15 
of the contours of the structures of interest constituted as 
vectorized two-dimensional images an extrapolation 
between the various sectional planes. 

The abovementioned extrapolation is carried out prefer- 
ably by means of a "B-spIine" type algorithm which seems 20 
best suited. This extrapolation transforms a discrete item of 
information, namely the successive sections obtained by 
means of the scanner analysis, into a continuous model 
permitting three-dimensional representation of the volume 
envelopes of the structures. 25 

It should be noted that the reconstruction of the volumes 
constituting the structures of interest introduces an approxi- 
mation related in particular to the spacing and non-zero 
thickness of the acquisition sections. An important charac- 
teristic of the invention, as explained in detail elsewhere, is 30 
on the one hand to minimize the resulting uncertainties in the 
patient-model correlation, and on the other hand to take into 
account the residual uncertainties. 

The CAD type software used possesses standard functions 
which enable the model to be manipulated in space by 35 
displaying it from different viewpoints through just a crite- 
rion defined by the surgeon (step 104). 

The software can also reconstruct sectional representation 
planes of the nonhornogeneous structure which differ from 
the planes of the images from the file 10, this making it 40 
possible in particular to develop knowledge enhancing the 
data for the representation by building up a neuro- 
anatomical map. 

The surgeon can next (step 105) determine a model of 
intervention strategy taking into account the modeled struc- 45 
tures of interest, by evaluating the distance and angle ratios 
on the two-and three-dimensional representations displayed. 

This intervention strategy will consist, in actual fact, on 
the one hand in localizing the tumor and in associating 
therewith a "target point", which will subsequently be able 50 
to substitute for the origin common to all the objects (real 
and images) treated by the system, and on the other hand in 
determining a simulated intervention path respecting to the 
maximum the integrity of the structures of interest. This step 
can be carried out "in the office", involving only the work- 55 
station. 

Once this operation is performed and prior to the 
intervention, the following phase consists in implementing 
the steps required to establish as exact as possible a corre- 
lation between the structure SNH (real world) and the 60 
representation RSNH (computer world). This involves steps 
106 to 109 of FIG. 5b. 

Firstly, as represented in FIG. 5b at step 107, marking of 
the base points of the reference structure SR with respect to 
the second reference frame is carried out with the aid of the 65 
marker means 3, by delivering to the system the coordinates 
X2, Y2, 72 of the said base points. 
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The following step 106 consists in identifying on the 
representations RSNH and RSR displayed on the screen the 
images of the base points which have just been marked. 
More precisely, with the aid of appropriate graphics 
peripherals, these representations (images) of the base points 
are selected one by one, the workstation supplying on each 
occasion (in this instance to the computer M02) the coor- 
dinates of these points represented in the reference frame R x . 

Thus the computer M02 has available a first set of 
three-dimensional coordinates representing the position of 
the base points in R 2 , and a second set of three-dimensional 
coordinates representing the position of the representations 
of the base points in R jt 

According to an essential feature of the invention, these 
data will be used to elaborate at 108, 109, tools for reference 
frame transfer (from R 1 to R 2 and vice versa) by calling 
upon an intermediate reference frame determined from the 
base points and constituting an intermediate reference frame 
specific to the reconstructed model. 

More precisely, the intermediate reference frame is con- 
structed from three base points selected so that, in this 
reference frame, the coordinates of the other base points 
after transfer from R 2 and the coordinates of the represen- 
tations of these other base points after transfer from R 2 are 
expressed with the greatest consistency and minimum dis- 
tortion. 

When the step of elaborating the reference frame transfer 
tools is concluded, these tools can be used by he system to 
secure optimal coupling between the real world and the 
computer world (step U10). 

Furthermore, according to a subsidiary feature of the 
present invention, the system can create on the display 
means a representation of the nonhornogeneous structure 
and of the intervention member which takes account of the 
deviations and distortions remaining after the "best" refer- 
ence frame transfer tools have been selected (residual 
uncertainties). More precisely, from these deviations can be 
deduced by the calculating means a standard error likely to 
appear in the mutual positioning between the representation 
of the nonhornogeneous structure and the representation of 
elements (tools, sighting axes, etc.) referenced on R 2 when 
using the reference frame transfer tools. This residual 
uncertainty, which may in practice be given substance 
through an error matrix, can be used for example to repre- 
sent certain contours (tool, structures of interest to be 
avoided during the intervention, etc.) with dimensions larger 
than those which would normally be represented starting 
from the three-dimensional data base or with the aid of 
coordinates marked in R 2 , the said larger dimensions being 
deduced from the "normal" dimensions by involving the 
error matrix. For example, if the member were represented 
normally, in transverse section, by a circle of diameter Dl, 
a circle of diameter D2>D1 can be represented in substance, 
with the difference D2-D1 deduced from the standard error 
value. In this way, when a direction of intervention will be 
selected making it possible to avoid traversing certain struc- 
tures of interest, the taking into account of an "enlarged" size 
of the intervention tool will eradicate any risk of the 
member, because of the abovementioned errors, accidently 
traversing these structures. 

Back at step 105, and as will be seen in more detail with 
reference to FIGS. 9a and 9b, the reference origin of 
intervention ORI and the direction of intervention A, that is 
to say the simulated intervention path, can be determined 
according to various procedures. 

According to a first procedure, the trajectory can be 
defined from two points, namely an entry point PE (FIG. 3) 
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and a target point, that is to say substantially the center of the The sub-module 1003 is itself followed, as represented in 

structure of interest consisting of the tumor to be observed FIG. 6, by a fourth sub-module 1004 for creating a unit 

or treated. Initially, these two points are localized on the three-dimensional orthogonal matrix N, characteristic of a 

model represented on the screen. right-handed orthonormal basis comprising three unit vec- 

According to a second methodology the trajectory can be s [ors -f, J, £, and which fa fied ^ second reference 

determined from the abovementioned arget point and from frame R ^ ^ fac( ^ ^ nonh neous struc . 

a direction which takes account of the types ot structures of . „ CTvT /v •„ „Z,- t : n,, .* j „ t uu t „ t L< „*u™ M 

, c . . ... . t , J * . t ture SNH is positionally tied with respect to this reference 

interest and of their positions with a view to optimally f rame 

respecting their integrity. ' ^. 

After the abovementioned step 108, the surgeon can at The three unit vectors i ', j *, k f are defined by the 

step 1110 perform the actual intervention. relations: 

The intervention can advantageously be performed by 

steering the tool or active member over the simulated ^ _^ _ 

intervention path, determined in step U10. j'-DEj\\DE\\ 

As a variant, given that the support arm 51 for the active 

member, equipped with its resolvcrs, continuously delivers 15 _ / _ ^, \ _ 

the coordinates in R 2 of the said active member to the 7 = 1 ED AEF |/|i£X>A£F|| 

system, it is also possible to perform the operation manually 1 I 

or semi-manually, by monitoring on the screen the position _ V - ' 

and motions of a representation of the tool and by comparing r= /" A £' 

them with the simulated, displayed intervention path. 20 As j nd j cated above , to the extent that the base points of 

It will furthermore be noted that the modeled direction of the re f ereri ce structure can be marked in R 2 with high 

intervention can be materialized with the aid of the laser precision, so their representation in the computer base R x is 

beam described earlier, the positioning of the latter (with marked with a certain margin of error given on the one hand 

respect to R 2 ) being likewise carried out by virtue of the the non-zero thickness (typically from 2 to 3 mm) of the 

reference frame transfer tools. 25 slices represented by the two-dimensional images from the 

Certain functional features of the system of the invention file 10, and on the other hand (in general to a lesser extent) 

will now be described in further detail with reference to the definition of each image element or pixel of a section. 

FIGS. 6, 7, 8, 9a and 9b. According to the invention, once a pair of transfer matri- 

Tbe module for elaborating the reference frame transfer ces M, N has been elaborated with selected points A, B, C, 

tools (steps 108, 109 of FIG. Sb) will firstly be described 30 E, F, it is sought to validate this selection by using one or 

with reference to FIG. 6. more additional base points; more precisely, for the or each 

This module comprises a first sub-module 1001 for additional base point, this point is marked in R 2 with the aid 

acquiring three points A, B, C, the images of the base points ° f th 5 P rot f 3 ?> the representation of this point is marked in 

of SR on the representation RSNH (the coordinates of these & after se ec f n 00 * e ^n and then the matrices N and 

points being expressed in the computer reference frame R,), 35 are a PP hed res P ectlvei y to the coordinates obtained, m 

by successive selections of these points on the representa- order to obtain their expressions in the bases ( i j k ') and 

tion To this effect, the surgeon is led, by means of a graphics ( ? J rcspectiveIV) If these expreS sions are in good 

interface such as a "mouse" to point successively at the three agreement, these two bases can be regarded as a single 

selected points A, B, C. intermediate reference frame, this securing the exact as 

The module for preparing the transfer tools also com- 40 possible mathematical coupling between the computer ref- 

prises a second sub-module, denoted 1002, for creating a erence frame Rj tied to the model and the "real" reference 

unit three-dimensional orthogonal matrix M, this matrix frame R*, tied to the patient, 

being characteristic of a right-handed orthonormal basis In practice, the module for elaborating the reference fxame 

represented by three unit vectors T, j\ k , which define an ttansfer tools can be de signed to perform steps 1001 to 1004 

intermediate reference frame tied to R r 45 iD succession on basic tnples which differ on each occasion 

_> (for example, if four base points have been defined associ- 

The unit vectors i , j and k are given by the relations: aied with four representations in RSR, there are four pos- 
sible triples), in order to perform the validation step 1005 for 

_+ -* each of these selections and finally in order to choose the 

j ~AB j\\ab\\ 50 triple for which the best validation is obtained, that is to say 

/ \ for which the deviation between the abovementioned 

expressions is smallest. This triple defines the "best plane" 
mentioned elsewhere in the description, and results in the 

\ / "best" transfer matrices M and N. 

7=7a7 55 ^ a var * ant > ft be possible for the selection of the best 

plane to be made at least in part by the surgeon by virtue of 
his experience. 

where || || designates the norm of the relevant vector. It should be noted that the reference frame transfer will 

In the above relations, the sign "A" designates the vector only be concluded by supplementing the matrix calculation 

product of the relevant vectors. 60 with the matrices M, N with a transfer of origin, so as to 

Similarly, the module for preparing the transfer tools create a new common origin for example at the center of the 

comprises a third sub-module, denoted 1003, for acquiring tumor to be observed or treated (point ORI). This transfer of 

three base points D, E, F, of the structure SR, these three origin is effected simply by appropriate subtraction of vec- 

poinls being those whose images on the model are the points tors on the one hand on the coordinates in R^ and on the 

A, B, C respectively. For this purpose, the surgeon, for 65 other hand on the coordinates in R 2 . These vectors to be 

example by means of the tactile lip 30, successively senses subtracted arc determined after localization of the center of 

these three points to obtain their coordinates in R 2 . the tumor on the representation. 
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Furthermore, the means described above for establishing From a simulated intervention path, for example on the 

the coupling between the patient's world and the model's representation RSNH, or from any other straight line 

world can also be used to couple to the model's world that selected by the surgeon, the invention furthermore enables 

of map data, also stored in the workstation and expressed in the model t0 be ^presented on the screen from a viewpoint 

a different reference frame denoted R 3 . In this case, since 5 corrcsponding t0 this straight Hne . XhlIS thc third transfcr 

these data contain* no specific visible mark, the earlier . . , , . n , n , 

described elaboration of matrices is performed by substilul- ^routine comprises, as represented m FIGS. 9a and 9b, a 

ing for these marks the positions of notable points of the farst module 301 tor visualizing the representation m a 

patient's head. These may be temporal points, the frontal direction given by two points and a second module 302 for 

median point, the apex of the skull, the center of gravity of visualizing the representation in a direction given by an 

the orbits of the eyes, etc. angle of elevation and an angle of azimuth. 

The corresponding points of the model can be obtained The first module 301 for visualizing the representation in 

either by selection by mouse or graphics tablet on the model, a direction given by two points comprises a first sub-module 

or by sensing on the patient himself and then using the denoted 3010 permitting acquisition of the two relevant 

transfer matrices. points which will define thc selected direction. The coordi- 

The above step of elaborating the reference frame transfer « ^ ^ ^ ^ 

tools, conducted m practice by the calculating means 4, . ...... • * • , 

makes it possible subsequently to implement the reference these P oints havin & either been acc ? mred Piously on the 

frame transfer means (FIGS. 7 and 8), nonhomogeneous structure SNH for example by means of 

With reference to FIG. 7, the first transfer sub-module 201 the tactile tip 30 and then subjected to the reference frame 

comprises a procedure denoted 2010 for aquiring the coor- 20 transfer, or chosen directly on the representation by means 

dinates XM, YM, ZM, expressed in K 19 of the point to be of the graphics interface of the "mouse" type, 

transferred, by selecting on the representation. The first sub-module 3010 is followed by a second 

The procedure 2010 followed by a procedure 2011 for sub-module denoted 3011 permitting the creation of a unit, 

calculating the coordinates XP, YP, ZP (expressed in R 2 ) of orthogonal three-dimensional matrix V characteristic of a 

the corresponding real point on the patient through the 15 , . . , , t . % , , ~r> . 

transformation* right-handed orthonormal basis i j ', k ,f the unit vectors 

{XP, YP, ZP}-M*N~ 3 *{XM, YM, ZM} where M * N~ 3 ?\ k", being determined through the relations: 
represents the product of the matrix M and the inverse 

matrix N. ^ _* _* 

The procedure 2011 is followed by a processing proce- 30 k" ~AB/\\AB\\; 

dure 2012 utilizing the calculated coordinates XP, YP, ZP, 7 M = -1" = 0;\\~r 11= i; 
for example to indicate the corresponding point on the 

surface of the structure SNH by means of the laser emission J " "* *" 
system EL, or again to secure the intervention at the relevant 

point with coordinates XP, YP, ZP (by steering the active 35 where "A" represents the vector product and " " symbolizes 

member). the scalar product. 

noSfT^' " ? rd T C ° *f « » frorr 1 SNH lo The sub . moduie 30 11 is followed by a routine 3012 

RSNH, the second sub-module 202 composes (FIG. 8) a maM u ib]e t0 secure for a]] the ints of the entities 

procedure denoted 2020 ifor acquiring on the structure SNH (structures of interest) of the three-dimensional data base of 

the coordinates XP,YP,ZP (expressed mR 2 ) of a point to be 40 coord i nales XW, YW, ZW in R, a conversion into the 
transferred. 

These coordinates can be obtained by means of the tactile orthonormal basis ( i H , j k n ) by the relation: 

tip 30 for example. The procedure 2020 is followed by a {XV, YV, ZV}«V*{XW, YW, ZW} 

procedure 2021 for calculating the corresponding coordi- The subroutine 3013 is 'then' followed by a subroutine 

nates XM, YM, ZM in K t through the transformation: 45 3014 for displaying the plane the subroutines 3013 and 

{XM, YM, ZM}=N*M" 1 *{XP, YP, ZP} 3014 being called up for all the points, as symbolized by the 

A procedure 2022 next makes it possible to effect the arrow returning lo block 3012 in FIG. 9a. 

displaying of the point with coordinates XM, YM, ZM on When all the points have been processed, an output 

the model or again of a straight line or of a plane passing module 3015 permits return to a general module, which will 

through this point and furthermore meeting other criteria. 50 be described later in the description, It is understood that this 

It will be noted here that the two sub-modules 201, 202 module enables two-dimensional images to be reconstructed 

can used [sic] by the surgeon at any moment for the purpose in planes perpendicular to the direction defined by A and B. 

of checking the valid nature of the transfer tools; in In the same way, thc second module 302 (FIG, 9b) for 

particular, it is possible to check at any time that a real base visualizing the representation from a viewpoint given by an 

point, with coordinates known both in R 2 and R 1 (for 55 angle of elevation and an angle of azimuth comprises a first 

example a base point of SR or an arbitrary notable point of sub-module 3020 for acquiring the two angles in the repre- 

the structure SNH visible on the images), correctly relocates sentation frame of reference. 

with respect to its image after transferring the coordinates in The selection of the angles of elevation and of azimuth 

step 2011. can be made by selecting from a predefined data base or by 

In the event of an excessive difference, a new step of 60 moving software cursers associated with each view or else 

elaboration of the transfer tools is performed. by modification relative to a current direction, such as the 

Furthermore, the sub-modules 201, 202 can be designed modeled direction of intervention. The sub-module 3020 is 

to also integrate the talcing into account of the residual itself followed by a second sub-module 3021 for creating a 

uncertainty, as spoken of above, so as for example to unit orthoganal three-dimensional matrix W characteristic of 

represent on the screen a point sensed not in a pointwise 65 . . . . , , . . . „ t> 

manner, but in the form for example of a circle or a sphere anght-handed orthonormal basis of unit vectors , •", j "', 

representing the said uncertainty. k m . They are defined by the relations: 
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k "' - z « sin(azimiUh) 

i • y = cos(elevation); 
/ "" • i"' = sin (elevation) 
7"' = I'" a7 ,m 

A routine 3022 is then called for ail the points of the 

entities of the three-dimensional data base of coordinates 

XW, YW, ZW and enables a first sub-routine 3023 to be 

called permitting calculation of the coordinates of the rel- 

— ► 

cvaot point in the right-handed orthonormal bases i '", 

j k m through the transformation: 

{XV, YV, ZV}=V*{XW, YW, ZW} 

The sub-routine 3023 is itself followed by a sub-routine 
3024 for displaying the plane i ,,! , j'", the two sub-routines 
3023 and 3024 then being called up for each point as 
symbolized by the return via the arrow to the block 3022 for 
calling the abovementioned routine. When all the points 
have been processed, an output sub-module 3025 permits a 
return to the general menu. 

Of course, all of the programs, sub-routines, modules, 
sub-modules and routines destroyed earlier are managed by 
a general "menu" type program so as to permit interactive 
driving of the system by screen dialogue with the interven- 
ing surgeon by specific screen pages. 

A more specific description of a general flow diagram 
illustrating this general program will now be given in 
connection with FIGS. 10a and 106. 

'ITius, in FIG. 10a has been represented in succession a 
screen page 4000 relating to the loading of data from the 
digitized file 10, followed by a screen page 4001 making it 
possible to secure the parameterizing of the grey scales of 
the display on the dynamic display means 1 and to calibrate 
the image, for example. 

The screen page 4001 is followed by a screen page 4002 
making it possible to effect the generation of a global view 
and then a step or screen page 4003 makes it possible to 
effect an automatic distribution of the sections on the screen 
of the workstation. 

A screen page 4004 makes it possible to effect a manual 
selection of sections and then a screen page 4005 makes it 
possible to effect the selection of the strategy (search for the 
entry points and for the possible directions of intervention, 
first localizing of the target (tumor . . . ) to be treated . . . ), 
as defined earlier, and to select the position and horizontal, 
sagittal and frontal distribution of the sections. 

A screen page 4006 also makes it possible to effect a 
display of the settings of a possible stereotaxic frame. 

It will be recalled that the reference structure SR advan- 
tageously replaces the stereotaxic frame formerly used to 
effect the marking of position inside the patient's skull. 

There may furthermore be provided a screen-page 4007 
for choosing strategic sections by three-dimensional 
viewing, on selection by the surgeon, and then at 4008 the 
aligning of the references of the peripherals (tool, sighting 
members, etc., with the aid of the probe 30. 

A screen page 4009 is also provided to effect the search 
for the base points on the patient with the aid of the said 
probe, following which the steps of construction of the 
reference frame transfer tools and of actual reference frame 
transfer are performed, preferably in a user-transparent 
manner. 
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Another screen page 4010 is then provided, so as to effect 
the localizing of the target on the representation (for 
example a tumor to be observed or treated in the case of a 
neurosurgical intervention) in order subsequently to deter- 
5 mine a simulated intervention path. 

Then a new screen page 4011 makes it possible to effect 
the setting of the guides for the tool on the basis of this 
simulated path before opening up the skin and bone flaps on 
the patient's skull. 

Then a new localizing step 4012 makes it possible to 
check whether the position of the guides corresponds cor- 
rectly to the simulated intervention path. 

The screen page 4012 is followed by a so-called inter- 
vention screen page, the intervention being performed in 
accordance with step 1110 of FIG. Sb. 
15 A more detailed description of the interactive dialogue 
between the surgeon and the system during a surgical, and 
in particular a neurosurgical, intervention will follow with 
reference to FIG. 10c and to all of the preceding description. 

The steps of FIG. 10c are also integrated in the general 
20 program mentioned earlier; there are undertaken in succes- 
sion a first phase I (preparation of the intervention), then a 
second phase II, (prior to the actual intervention, the patient 
is placed in a condition for intervention, the reference 
structure SR being tied to the second reference frame R 2 ), 
25 then a third phase III (intervention) and finally a post- 
intervention phase IV. 

With a view to preparing the intervention, the system 
requests the surgeon (step 5000) to choose the elementary 
structures of interest (for example bones of the skull, 
ventricles, vascular regions, the tumor to be explored or 
treated, and the images of the marks constituting in the first 
reference frame the representation RSR). 

The choice of the elementary structures of interest is made 
on the display of the tomographic images, for example, 
called up from the digitized file 10. 
35 The system next performs, at step 5001, a modeling of the 
structures of interest, as described earlier. Then, the nonho- 
mogencous structure having been thus constituted as a 
three-dimensional model RSNH displayed on the screen, the 
intervening surgeon is then led to perform a simulation by 
40 three-dimensional imaging, at step 5002, with a view to 
defining the intervention path of the tool 50. 

During phase II the patient being placed in a condition for 
intervention and his head and the reference structure SR 
being tied to the second reference frame R 2 , the surgeon 
45 performs at step 5003 a search for the position of the marks 
Ml to M4 constituting base points of the reference structure 
in the second reference frame R 2 , and then during a step 
5004, performs a search for the position of the sighting 
systems, visualizing member OV, or of the tools and inter- 
so vention instruments 50, still in the second reference frame 
R 2 , so as, if appropriate, to align these implements with 
respect to R 2 . 

The system then performs the validation of the 
intervention/patient spaces and representation by three- 

55 dimensional imaging in order to determine next the common 
origin of intervention ORI. In other words, the matrix 
reference frame transfer described above is supplemented 
with the necessary origin translations (origins 01 and 02 
aligned on ORI). 

60 This operation is performed as described earlier. 

Phase III corresponds to the intervention, during which 
the system effects at step 5006 a permanent coupling in real 
lime between the direction of aim of the active member 50, 
and/or of the direction of aim of the sighting member OV 

65 (and if appropriate of the laser beam), with the direction of 
aim (of observation) simulated by three-dimensional imag- 
ing on the display means 1, and vice versa. 
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In the following step 5007, the coupling is effected of the 
movements and motions of the intervention instrument with 
their movements simulated by three-dimensional imaging, 
with automatic or manual conduct of the intervention. 

As noted at 5008, the surgeon can be supplied with a 
permanent display of the original two-dimensional sectional 
images in planes specified with respect to the origin ORI and 
to the direction of intervention. Such a display enables the 
surgeon at any time to follow the progress of the intervention 
in real time and to be assured that the intervention is 
proceeding in accordance with the simulated intervention. In 
phase IV which is executed after the intervention, the system 
effects a saving of the data acquired during the intervention, 
this saving making it possible subsequently to effect a 
comparison in real time or deferred in the event of succes- 
sive interventions on the same patient. 

Furthermore, the saved data make it possible to effect a 
playback of the operations carried out with the option of 
detailing and supplementing the regions traversed by the 
active member 50. 

Thus, a particularly powerful interactive system for local 
intervention has been described. 

Thus, the system which is the subject of the present 
invention makes it possible to represent a model containing 
only the essential structures of the nonhomogeneous 
structure, this facilitating the work of preparation and of 
monitoring of the intervention by the surgeon. 

Moreover, the system, by virtue of the algorithms used 
and in particular by minimizing the distortion between the 
real base points and their images in the 2D sections or the 
maps, makes it possible to establish a two-way coupling 
between the real world and the computer world through 
which the transfer errors are minimized, making possible 
concrete exploitation of the imaging data in order to steer the 
intervention tool. 

To summarize, the system makes possible an ineractive 
[sic] medical usage not only to create a three-dimensional 
model of the nonhomogeneous structure but also to permit 
a marking in real time with respect to the internal structures 
and to guide the surgeon in the intervention phase. 

More generally, the invention makes it possible to end up 
with a coherent system in respect of: 

the two-dimensional imaging data (scanner sections, 
maps, etc.) 

the three-dimensional data base; 

the data supplied by the marker means 3 in the reference 
frame R 2 ; 

the coordinate data for the sighting systems and interven- 
tion tools; 

the real world of the patient on the operating table. 
Accordingly, the options offered by the system are, in a 
non-limiting manner, the following: 

the tools and of [sic] their position can be represented on 
the screen; 

the position of a point on the screen can be materialized 
on the patient for example with the aid of the laser 
emission device EL; 

the orientation and the path of a tool such as a needle can 
be represented on the screen and materialized on the 
patient optically (laser emission) or mechanically 
(positioning of the guide-arm in which the tool is 
guided in translation): 

an image of the patient, yielded for example by a system 
for talcing pictures if appropriate in relief, can be 
superimposed on the three-dimensional representation 
modeled on the screen; thus, any change in the soft 
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external parts of the patient can be visualized as com- 
pared with the capture by the scanner; 

it being possible for the surgeon's field of view given by 
a sighting member (such as a surgical microscope) to be 
referenced with respect to R 2 , the direction of visual- 
ization of the model on the screen can be made identical 
to the real sight by the sighting member; 

finally, the three-dimensional images, normally displayed 
on the screen in the preceding description, may as a 
variant be introduced into the surgeon's microscope so 
as to obtain the superposition of the real image and the 
representation of the model. 

We claim: 

1, An interactive system for local intervention inside a 
region of a non-homogeneous structure to which is con- 
nected a reference structure containing a plurality of base 
points, the interactive system comprising: 

means for dynamically displaying a three-dimensional 
image of a representation of the non-homogeneous 
structure and of the reference structure connected to the 
non-homogeneous structure, wherein the three- 
dimensional image also includes a plurality of images 
of the plurality of base points; 

means for determining a set of coordinates of the plurality 
of images of the plurality of base points in a first 
reference frame; 

means for fixing a position of the non-homogeneous 
structure and of the reference structure with respect to 
a second reference frame; 

means for determining a set of coordinates of the plurality 
of base points in the second reference frame; 

means of intervention comprising an active member 
whose position is determined with respect to the second 
reference frame; 

means for generating a plurality of reference frame trans- 
lation tools for translating a plurality of reference 
frames from the first reference frame to the second 
reference frame and vice versa, based on the set of 
coordinates of the plurality of images of the plurality of 
base points in the first reference frame and of the set of 
coordinates of the plurality of base points in the second 
reference frame, in such a way as to reduce to a 
minimum at least one of a set of deviations between the 
set of coordinates of the plurality of images of the 
plurality of base points in the first reference frame and 
the set of coordinates of the base points, expressed in 
the first reference frame using the plurality of reference 
frame translation tools; 

means for defining, with respect to the first reference 
frame, a simulated origin of intervention and a simu- 
lated direction of intervention; and, 

means for transferring the plurality of reference frames 
using the plurality of reference frame translation tools 
to establish a bidirectional coupling between the simu- 
lated origin of intervention and the simulated direction 
of intervention and the position of the active member. 

2. The interactive system according to claim 1, wherein 
the plurality of reference frame translation tools comprise: 

means for creating a matrix (M) for transferring between 
the first reference frame and a first intermediate refer- 
ence frame based on a set of coordinates of a set of 
three images of a set of three base points of the 
reference structure; 

means for creating a matrix (N) for transferring between 
the second reference frame and a second intermediate 
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reference frame based on the set of coordinates of the 
set of three images of the set of three base points of the 
reference structure; and, 
means for validating matrix (M) and matrix (N) based on 
the set of three base points and the set of three images, 5 
such that at least one deviation between an expression 
for at least one additional base point in the second 
intermediate reference frame and an expression for at 
least one image point of the additional base point in the 
first intermediate reference frame is reduced to a mini- 10 
mum. 

3. The interactive system according to plurality of claim 
2, wherein the means for transferring the reference frames 
using the plurality of reference frame translation tools 
further comprises: 15 

a first transfer sub-module for transferring a set of 
rep resent at ion/no n -homogeneous structure 
coordinates, and 

a second transfer sub-module for transferring a set of 0Q 
non-homogeneous structure/representation coordi- 
nates. 

4. The interactive system according to claim 3, wherein 
the first transfer sub-module comprises: 

means for acquiring a set of coordinates (XM, YM, ZM), 2 5 
expressed in the first reference frame, of a point of the 
representation of the non-homogeneous structure to be 
transferred, by selection on the representation; 

means for calculating a set of corresponding coordinates 
(XP, YP, ZP), expressed in the second reference frame, 30 
on the non-homogeneous structure through a transfor- 
mation: 

{YP,YP, ZP^M*^ 1 *{XM,YM,ZM} where M * N" 1 
represents a product of the matrix (M) and an inverse 
of the matrix (N), and 35 

means for processing, with the aid of the corresponding 
coordinates (YP, YP, ZP), to display a corresponding 
point on a surface of the non-homogeneous structure 
and to secure the intervention. 

5. The interactive system according to claim 3, wherein 40 
the second transfer sub-module comprises: 

means for acquiring a set of coordinates (XP, YP, ZP), 
expressed in the second reference frame, of a point of 
the non-homogeneous structure to be transferred; 45 

means for calculating a set of corresponding coordinates 
(XM YM, ZM), expressed in the first reference frame, 
of the representation through a transformation: 

{YM, YM, ZM}-N*M~ 1 *{XP,ZP,ZP} where N*M~ 3 
represents the product of the matrix (N) and an inverse 50 
of the matrix (M); and, 

means for displaying the representation using the set of 
corresponding coordinates (YM, YM, ZM). 

6. The interactive system according to claim 1, wherein 
the means for generating the plurality of reference frame 55 
translation tools also generate, in association with the ref- 
erence frame translation tools, tools for taking into account 

a residual uncertainty which is based on the set of deviations 
between the set of coordinates of the plurality of images of 
the plurality of base points in the first reference frame and 60 
the set of coordinates of the base points, the tools for taking 
into account the residual uncertainty usable for displaying a 
set of contours in the representation whilst taking into 
account the residual uncertainties. 

7. The interactive system according to claim 1, wherein 65 
the means of dynamic displaying the three-dimensional 
image comprises: 
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a file containing digitized data from a set of two- 
dimensional images constituted by successive non- 
invasive tomographic sections of the non- 
homogeneous structure; 

means for calculating and reconstructing the three- 
dimensional image from the set of two-dimensional 
images; and 

a high-resolution display screen. 

8. The interactive system according to claim 7, wherein 
the means for calculating and reconstructing the three- 
dimensional image from the set of two-dimensional images 
comprises a program consisting of computer-aided design 
type software. 

9. The interactive system according to claim 1, wherein 
the means for determining the set of coordinates of the 
plurality of base points in the second reference frame 
comprises a three-dimensional probe equipped with a tactile 
tip for delivering a set of coordinates of the tactile tip in the 
said second reference frame. 

10. The interactive system according to claim 1, wherein 
the means for determining the set of coordinates of the 
plurality of base points is the second reference frame com- 
prises at least one of a set of optical sensors and a set of 
electromagnetic sensors. 

11. The interactive system according io claim 1, wherein 
a portion of the set of the plurality of base points of the 
reference structure comprises a plurality of marks positioned 
on a lateral surface of the non-homogeneous structure. 

12. The interactive system according to claim 11, wherein 
the plurality of marks are four in number and are distributed 
over the lateral surface so as to define a substantially 
symmetrical tetrahedron. 

13. The interactive system according to claim 1, wherein 
the means of intervention comprises: 

a guide arm to secure intervention in the region of the 
non-homogeneous structure, the guide arm having a 
position marked with respect to the second reference 
frame; and, 

an active intervention member whose position is marked 
with respect to the second reference frame. 

14. The interactive system according to claim 13, wherein 
the active intervention member is removable and selected 
from the group consisting of: 

tools for trephining; 
needles and implants; 

laser and radioisotope emission heads; and, sighting and 
viewing systems. 

15. The interactive system according to claim 1, wherein 
the means for transferring the plurality of reference frames 
establishes a coupling between a direction of visualization of 
the representation of the non-homogeneous structure on the 
display means and a direction of observation of the non- 
homogeneous structure and of the reference structure by the 
active intervention member. 

16. The interactive system according to claim 15, further 
comprising: 

a first module for visualizing a representation in a direc- 
tion given by two points; 

a second module for visualizing a representation in a 
direction given by an angle of elevation and an angle of 
azimuth. 

# * * * 
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Miiorney l/uckoI. ou/^m-uuuu io/ncrt 
First Named inventor: Jean Francois Uhl 
Application Number: 09/784,829 
Filing Date: February 8, 2001 
Art Unit: 3737 

Examiner Name: Ruth S. Smith 


iNumero oe reyisire. ou/^+m-uuuu lo/ricH 
Norn du premier inventeur: Jean Francois Uhl 
Numero de ['application: 09/784,829 
Date de depot: 8 fevrier 2001 
Unite d'art : 3737 

Norn de I'examinateur: Ruth S. Smith 




I/We hereby declare that: 

Every error in the patent which was corrected in the present reissue 
application, and which is not covered by the prior oath(s) and/or 
deciaration(s) submitted in this application, arose without any 
deceptive intention on the part of the applicant. 

WARNING: 

Petitioner/applicant is cautioned to avoid submitting personal 
information in documents files in a patent application that may 
contribute to identity theft. Personal information such as social 
security numbers, bank account numbers, or credit card numbers 
(other than a check or credit card authorization form PTO-2038 
submitted for payment purposes) is never required by the USPTO to 
support a petition or an application. If this type of personal 
information is included in documents submitted to the USPTO, 
petitioners/applicants should consider redacting such personal 
information from the documents before submitting them to the 
USPTO. 

Petitioner/application is advised that the record of a patent 
application is available to the public after publication of the 
application (unless a non-publication request in compliance with 37 
CFR 1.213(a) is made in the application) or issuance of a patent. 
Furthermore, the record from an abandoned application may also be 
available to the public if the application is referenced in a published 
application or an issued patent (see 37 CFR 1.14). Checks and 
credit card authorization forms PTO-2038 submitted for payment 
purposes are not retained in the application file and therefore are 
not publicly available. 

I/We hereby declare that all statements made herein of my/our own 
knowledge are true and that all statements made on information and 
belief are believed to be true; and further that these statements were 
made with the knowledge that willful false statements and the like 
so made are punishable by fine or imprisonment, or both, under 18 
U.S.C. 1001 and that such willful false statements may jeopardize 
the validity of the application or any patent issued thereon. 


Je(nous) declare(ons) que : 

Toute erreur dans ie brevet qui a ete corrigee dans cette demande de redelivrance 
et qui n'est pas couverte par le present serment ou declaration soumis a cette 
demande, s'est produite sans intention de tromper de la part de I'appliquant. 

AVERTISSEMENT : 

Le requerant / demandeur est mis en garde contre la soumission de 
renseignements personnels dans les documents deposes dans une demande de 
brevet qui peuvent aider a I'usurpation d'identite. Les renseignements personnels 
tels que numeros de securite sociale, numeros de compte bancaire, ou numeros 
de carte de credit (autre qu'un cheque ou un formulaire d'autorisation de carte de 

pff*ri£f Harm Ip hut Hp fairp un naipmpntl np *;nnt iamaiQ rpniii«s nar 
I'USPTO (Bureau des brevets des Etats-Unis) pour appuyer une petition ou une 
demande. Si ce type de renseignements personnels est inclus dans les documents 
deposes a rUSPTO, les demandeurs / requerants devraient envisager de les 
enlever des documents avant de les soumettre a I'USPTO. 

Le requerant / demandeur est informe que le dossier de demande de brevet est a la 
disposition du public apres la publication de ta demande (sauf si une demande de 
non-publication en conformite avec 37 CFR 1.213 (a) a ete faite dans cette 
demande) ou apres ta delivrance d'un brevet. En outre, le dossier d'une demande 
abandonnee peut egalement etre mis a la disposition du public si la demande est 
referencee dans une application publiee ou un brevet delivre (voir 37 CFR 1.14). 
Les cheques et formulaires d'autorisation de carte de credit PTO-2038 soumis pour 
fins de paiement ne sont pas conserves dans le dossier de demande et ne sont 
done pas accessibles au public. 

Je / Nous declarons que toutes les declarations faites selon ma/not re 
connaissance dans ce document sont veridiques et que toutes les declarations 
faites sur des informations et croyances sont considerees comme vraies. De plus, 
ces declarations ont ete faites en sachant que toute fausse declaration volontaire 
est passible d'une amende ou d'une peine d'emprisonnement, ou les deux, en 
vertu de la loi 18 USC 1001 et que de telles declarations risquent de compromettre 
la validite de la demande ou d'un brevet delivre a partir de celle-ci. 


Name of First Inventor: Jean Francois Uhl 
Inventor's Sianature 


Norn du premier inventeur: Jean Francois Uhl 
Sianature de I'inventeur 


Date 


Date 








Name of Second Inventor: Joel Henrion 
inventor's Sianature 


Norn du deuxieme inventeur : Joel Henrion 
Sianature de Tinventeur 


Date 


Date 








Name of Third Inventor: Michel Scriban 
Inventor's Sianature 


Norn du troisieme inventeur : Michel Scriban 
Sianature de ilnventeur 


Date 


Date 








Name of Fourth Inventor: Jean-Baptiste Thiebaut 
inventor's Sianature 


Norn du quatrieme inventeur : Jean-Baptiste Thiebaut 
Sianature de i'inventeur 


Date 


Date 
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ATTACHMENT O 



I 



i 



Taylor, Michael 



Sent: 

To: 

Cc: 



Subject: 



From: 



Sophie PONCET <sophie.ccgl@wanadooir> 
Tuesday, November 29, 2011 6:07 AM 
Taylor, Michael 
GRANGE Maxime 

Re: MEDTRONIC (5074A-000013/REA) [HDP-Troy_Legal.FID2452556] 



Importance: 



High 



Cher Confrere, 



J'ai bien re?u vos deux derniers envois concernant le dossier en reference. 
Je maintiens les termes de ma correspondance du 15 novembre 2011. 

En effet, les specialistes frangais que je represente ne peuvent en aucun cas etudier, de maniere gracieuse, 
I'integralite des pieces que vous leur avez transmises. 

II n'est pas non plus possible qu'ils donnent leur signature sur des documents qui n'auraient pas pu etre 
prealablement etudies. 

Je reste done dans I'attente, a nouveau, de connaitre la position definitive que prendra votre cliente, la societe 
MEDTRONIC. 

Vous remerciant de me fixer, 

Je vous souhaite bonne reception de la presente, et vous prie de croire, en mes sentiments les plus devoues. 

Maxime GRANGE 

ACCESS AVOCATS 

63 avenue de Saxe - 69003 LYON 

T 04 72 84 99 60 

F 04 72 84 99 69 

Original Message 

From: Taylor, Michael 
To: 'Sophie PONCET' 

Cc: 'GRANGE Maxime' ; Warner Rick ; Neal, Patrick ; Hall, Stephanie 
Sent: Wednesday, November 23, 2011 3:21 PM 

Subject: RE: MEDTRONIC (5074A-000013/REA) [HDP-Troy„Legal.FID2452556] 
Cher GRANGE, 

My email of yesterday appears to have been missing our letter. I apologize for this oversight. If you received our letter 
dated November 22, 2011 in yesterday's email, this email is a duplicate. 



Best Regards, 
Michael Taylor 



i I yv k I p»'c:c: liSlMi Michael L. Taylor j Patent Attorney 

r-i.^wmt^ i &. ; 3aaaa ; o | 248.641.1600 F | 248.641.0270 D | 248.641.1289 

i^^^^l DICKE V ip causes Worldwide 



From: Taylor, Michael 

Sent: Tuesday, November 22, 2011 11:57 AM 
To: 'Sophie PONCET 

Cc: GRANGE Maxime; Warner, Rick; Neal, Patrick; Hall, Stephanie 

Subject: RE: MEDTRONIC (5074A-000013/REA) [HDP-TroyJ.egal.FID2452556] 

Cher GRANGE, 

We understand that you are an attorney for all of Michel Scriban, Joel Henrion, Jean Francois UHL, and Jean-Baptiste 
Thiebaut. 

Please see our attached letter dated November 22, 2011, which refers to our previous correspondence, also attached. 
Best Regards, 
Michael Taylor 



UI'AHMP*CQ Michael L.Taylor | Patent Attorney 

K^.:=3£s> mmmmm Q 2 48,641.1600 F | 248.641.0270 D j 248.641.1289 
— 8 DICKEY ip causes Worldwide 



From: Sophie PONCET [mailto:sophie.ccgl@wanadoo.fr] 

Sent: Tuesday, November 15, 2011 9:28 AM 

To: Taylor, Michael 

Cc: GRANGE Maxime 

Subject: MEDTRONIC 

Importance: High 



Cher Monsieur, 



Maitre GRANGE, avocat au barreau de Lyon, repond a vos correspondances echangees notamment avec 
Michel SCRIBAN concernant le dossier en reference. 

Nous restons dans I'attente de votre reponse. 

Vous en remerciant par avance, 

Nous vous adressons nos sentiments les plus devoues. 



2 



Maxime GRANGE 



Access Avocats 

63 avenue de Saxe - 69003 LYON 
T 04 72 84 99 60 
F 04 72 84 99 69 



HARNESS, DICKEY & PSEKCE, PLC LEGAL NOTICE; The information contained in this transmission is intended only for the individual(s) or entity(ies) to 
whom it is addressed, it may contain information protected from use and/or disclosure by law, including information that is protected as confidential, attorney- 
client privileged, attorney work product and/or trade secrets, If the reader of this message is not the intended recipient, or an employee or agent responsible for 
delivering this message to the addressee, the reader is hereby notified that any use. distribution or copying of this communication is strictly prohibited. If you 
believe you have received this facsimile or message in error, please immediately notify us at our expense by return mail or e-mail and permanently delete or 
destroy all copies of the message. 

To comply with U.S. Treasury regulations, we advise you that any discussion of Federal tax issues in this communication was not intended to be used, and 
cannot be used, by any person (i) to avoid penalties that may be imposed by the Interna! Revenue Service, or (it) to promote, market or recommend to another 
party any matter addressed herein. 

Any portions of this transmission containing controlled technical data are restricted by U.S. export laws and regulations, and may not be distributed or 
retransmitted to non-U. S. persons without appropriate licensing or a licensing exemption. Neither this information block, nor the signature block, nor the typed 
name of the sender, nor anything else in this message is intended to constitute an electronic signature unless a specific statement to the contrary is included. 
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ATTACHMENT P 



! 



Taylor, Michael 



From: 
Sent: 
To: 
Cc: 

Subject: 
Attachments: 



Global Language links <globallanguagelinks@yahooxom> 
Tuesday, November 29, 2011 11:00 AM 
Taylor, Michael 

kwallace@globlanglinksxom; Laura 

MEDTRONIC (5074A-000013/REA) [HDP-TroyJ_egal.FID2452556] 
image001.gif 



Hi Micheal, Here it is. 




Thank you and have a wonderful day! ! 



Dear Colleague, 

I received your last two correspondences regarding the file in reference. 
I uphold what I stated in my correspondence of November 15 th , 2011. 

Indeed, the French experts that I represent cannot, in any way, study gracefully all the documents you have 
provided them with. 

Moreover, it is not possible for them to sign documents that they did not have the chance to previously study. 
Again, I am waiting for the decision that your client, Medtronic, will take. 
Thank you for letting me know this decision and settling in this matter, 
I wish you good reception of this letter. Sincerely, 



Forwarded Message — 

From: Kimberly Wallace <kwallace@globlanglinks.com> 
To: gll <globallanguagelinks@yahoo.com> 
Sent: Tuesday, November 29, 2011 10:21 AM 

Subject: Fwd: FW: MEDTRONIC (5074A-000013/REA) [HDP-Troy_Legal.FID2452556] 



Sent from my MetroPCS Android Device 
Original Message 

Subject: FW: MEDTRONIC (5074A-000013/REA) [HDP-Troy_Legal.FED2452556] 
From: "Taylor, Michael" <mltavlor@HDP.com > 

To: f " KWallace@GlobLangLinks.com ' " <KWallace@GlobLangLinks.com> 
CC: 

Dear Kim, 

Is there a chance I can get the below translated to English today? 



Thanks, Michael 



Cher Confrere, 



J'ai bien re<?u vos deux derniers envois concemant le dossier en reference. 
Je maintiens les termes de ma correspondance du 15 novembre 201 1. 

En effet, les specialistes fran9ais que je represente ne peuvent en aucun cas etudier, de maniere gracieuse, 
Fintegralite des pieces que vous leur avez transmises. 

II n'est pas non plus possible qu'ils donnent leur signature sur des documents qui n'auraient pas pu etre 
prealablement etudies. 

Je reste done dans l'attente, a nouveau, de connaitre la position definitive que prendra votre cliente, la societe 
MEDTRONIC. 

Vous remerciant de me fixer, 

Je vous souhaite bonne reception de la presente, et vous prie de croire, en mes sentiments les plus devoues. 



[Description: HDP]<htt p://www.hdp,com/ > 

Michael L. Taylor | Patent Attorney 

O | 248.641.1600 F | 248.641.0270 D | 248.641.1289 

IP Causes Worldwide 



HARNESS, DICKEY & PIERCE, PLC LEGAL NOTICE: The information contained in this transmission is 
intended only for the individual(s) or entity(ies) to whom it is addressed. It may contain information protected 
from use and/or disclosure by law, including information that is protected as confidential, attorney-client 
privileged, attorney work product and/or trade secrets. If the reader of this message is not the intended recipient, 
or an employee or agent responsible for delivering this message to the addressee, the reader is hereby notified 
that any use, distribution or copying of this communication is strictly prohibited. If you believe you have 
received this facsimile or message in error, please immediately notify us at our expense by return mail or e-mail 
and permanently delete or destroy all copies of the message. 

To comply with U.S. Treasury regulations, we advise you that any discussion of Federal tax issues in this 
communication was not intended to be used, and cannot be used, by any person (i) to avoid penalties that may 
be imposed by the Internal Revenue Service, or (ii) to promote, market or recommend to another party any 
matter addressed herein. 

Any portions of this transmission containing controlled technical data are restricted by U.S. export laws and 
regulations, and may not be distributed or retransmitted to non-U. S. persons without appropriate licensing or a 
licensing exemption. Neither this information block, nor the signature block, nor the typed name of the sender, 
nor anything else in this message is intended to constitute an electronic signature unless a specific statement to 
the contrary is included. 
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